Abstract

In view of the fact that the classical fast motion estimation methods are easy to fall into local optimum and suffer the high computational cost, the convergence of the motion estimation method based on the swarm intelligence algorithm is very slow. A new block motion estimation method based on single-dimension perturbation glowworm swarm optimization algorithm is proposed. Single-dimension perturbation is a local search strategy which can improve the ability of local optimization. The proposed method not only has overcome the defect of falling into local optimum easily by taking use of both the global search ability of glowworm swarm optimization algorithm and the local optimization ability of single-dimension perturbation but also has reduced the computation complexity by using motion vector predictor and terminating strategies in view of the characteristic of video images. The experimental results show that the performance of the proposed method is better than that of other motion estimation methods for most video sequences, specifically for those video sequences with violent motion, and the searching precision has been improved obviously. Although the computational complexity of the proposed method is slightly higher than that of the classical methods, it is still far lower than that of full search method.

1. Introduction

Motion estimation (ME) is the prediction of the motion vector (MV) between the current frame and the reference frame. As an important part of video processing, it has been widely used in compression, computer vision, target tracking, and monitoring of industrial and other areas. Among many methods of ME, block matching algorithm (BMA) has been accepted by many video compression coding standards due to its simple structure and ease of implementation, for example, MPEG-1, MPED-4, H.261, H.263, and H.264. Full search (FS) [1] algorithm is the simplest and the most accurate BMA, but its high computational complexity makes it not suitable for real-time implementation, so a number of fast BMAs have been proposed. The most widely used of them is the BMA based on template, the classic representatives including three-step search (TSS) [2], new three-step search (NTSS) [3], four-step search (FSS) [4], diamond search (DS) [5], and the improvement based on these methods like hexagon search (HEXBS) [6] and directional diamond search (DDS) [7]. In addition, adaptive rood pattern search (ARPS) [8] which is the classic representative of the BMA based on motion vector predictor has a superior performance. However, all above methods rely on the unimodal error surface assumption (UESA), which is usually not established for the real video sequences. Therefore, all these algorithms will get trapped into local optimum easily, especially under the condition of a complex movement. In order to solve this problem, some BMAs based on global optimal are proposed in recent years, such as fast ME method based on genetic algorithm (GA) [9], block motion estimation based on immune clonal selection (BMEICS) [10], BMA based on particle swarm optimization for ME [11, 12], and so forth. These methods have a good global optimization ability which can overcome the defect of falling into local optimum easily to a certain extent, but it also loses the ability of local optimization, resulting in slow convergence speed and high computational complexity.

Glowworm swarm optimization (GSO) [13] algorithm presented by Krishnanand and Ghose is a new kind of swarm intelligence algorithm. It is inspired by social behavior of the glowworm and the phenomenon of bioluminescent communication. GSO algorithm has the characteristics of ant colony optimization (ACO) algorithm and artificial bee colony (ABC) algorithm, but compared to ACO, GSO algorithm has better generality and a wider range of application. Although the roulette wheel method is applied to choose the moving object in ABC algorithm as well as GSO algorithm, all scout bees are candidate moving objects of the onlooker Bee, which makes it easy to fall into local optimum. In GSO algorithm, each glowworm selects its moving object only within its neighborhood (a collection of the neighbor glowworms who have higher luciferin than the current glowworm). Since different glowworms have different neighborhoods, the choice of moving object is different too, which can avoid falling into local optimum. Therefore, GSO algorithm has a good performance in solving multiobjective optimization problems. It has been successfully applied to the multisource issue tracking and location problem, harmful gas leak location problem, solving the nonlinear equations [14], clustering analysis [15], traveling salesman problem [16], 0-1 knapsack problem [17], constrained engineering design problem [18], the noise of the sensor test, and so forth. So far, it still has the problems of slow convergence speed and weak local search ability as well as other swarm intelligent algorithms. For video processing, on the premise of high precision strong ability of local search, fewer iteration times and faster convergence speed are very important for reason of huge image data. According to these defects of original GSO algorithm and the characteristics of ME, single-dimension perturbation glowworm swarm optimization (SDGSO) algorithm is proposed by introducing the local search strategy called single-dimension disturbance and applied to ME. In simulation experiments, the proposed method has been compared to other ME methods, including classic fast BMAs and the BMAs based on swarm intelligent algorithms.

2. Single-Dimension Perturbation Glowworm Swarm Optimization Algorithm

In GSO, the agents are thought of as glowworms that carry a luminescent quantity called luciferin along with them. The glowworms encode the fitness of their current locations, evaluated using the objective function, into a luciferin value that they broadcast to their neighbors. The glowworm identifies its neighbors and computes its movements by exploiting an adaptive neighborhood, which is bounded above by a radial sensor range. Each glowworm selects, using a probabilistic mechanism, a neighbor that has a luciferin value higher than its own and moves toward it. These movements, based only on local information and selective neighbor interactions, enable the swarm of glowworms to partition into disjoint subgroups that converge on optima, but it also means that most glowworms will get together in the late iteration, which will lead to a slow optimum speed or even fall into local optimum. There is a typical example to illustrate the gathering process of GSO algorithm as showed in Figure 1(a). There are two neighbors and within the local-decision domain of glowworm , and we suppose that glowworm has chosen glowworm as its neighbor in the probability , so it moves to at a step size. There is only one neighbor within the local-decision domain of glowworm , so it moves to at a step size. After many times of iteration, glowworms and will always move along the direction of glowworm , and converge to the location of glowworm finally as shown in Figure 1(b). However, this location may be a local optimum, rather than the global optimal. We suppose that the luciferin value of glowworm is bigger than that of glowworm as shown in Figure 1(c), but because glowworm cannot find glowworm within its local-decision domain, it will always move along the direction of glowworm and fall into local optimum in the end. Therefore, we consider disturbance after each move, glowworm is taken for instance, and it can be found that the location of glowworm is better than that of by means of the disturbance, and the glowworm is within the local-decision domain of , and then glowworm will move to at a probability, which will change the direction of the movement and avoid falling into local optimum.

In this section, the basic glowworm swarm optimization algorithm and single-dimension perturbation strategy are introduced first, and then a novel glowworm swarm optimization algorithm based on single-dimension perturbation is proposed.

2.1. Basic Glowworm Swarm Optimization Algorithm

Here is the description of basic GSO algorithm.

In Luciferin-update phase, the luciferin updating of each glowworm depends on its current fitness value. That is to say, current luciferin value needs to add up a proportion of its current fitness value on the basis of its value at the previous moment. Besides, the luciferin value which is volatilized as the passage of time needs to be subtracted. The specific luciferin update formula can be stated as where is the luciferin value of glowworm , is the volatile coefficient of luciferin, is the update coefficient, and is the objective function value of glowworm .

In movement phase, neighbor glowworms which have higher luciferin value than its own are selected by each glowworm in its local-decision domain and form a neighborhood, and the moving object will be chosen from the neighborhood according to the probabilistic mechanism. For the glowworm , probabilistic equation of moving to a neighbor is given as follows: where , is the neighborhood of glowworm .

Movement equation is where is the location of glowworm at th iteration, represents the step size, and is the Euclidean operator.

In local-decision domain update rule, during the course of iteration, the number of neighbors is dynamic for each agent, so the local decision range must be adjusted to ensure that the swarm of glowworms can capture multiple peaks. The update rule is given by where is the perceive domain, is a constant factor, is a modulus used to control the number of neighbors, and is the number of neighbor glowworms.

2.2. Local Search Strategy

Though GSO algorithm has many advantages such as strong universality, good overall and good performance at solving multi-objective optimization, it also has some shortcomings as well as other swarm intelligent algorithms, such as slow convergence speed and poor local search ability. According to these problems, a novel local search strategy called single-dimension perturbation search (SDPS) [19] is introduced in this paper. SDPS is a local search strategy which is applied after the movement phase according to (3). With the good global optimization ability of GSO algorithm, SDPS will enhance the ability of local optimization, speed up the convergence, and improve the precision.

In 2D space, for instance, the four points which distance the glowworm steps will be selected randomly after the location updating, and the new location will be calculated at a certain probability. If the objective function value of the new location is bigger than that of the current location, move or else, keep still.

2.3. Main Steps of the SDGSO Algorithm

The computational procedure of the proposed algorithm can be summarized as follows.

Step 1. Initialize the population. A swarm of glowworms is generated randomly within the search space: dimensions of the search space—, initial location—, initial luciferin—, perceive domain—, maximum iterations—, step size—, step size of local search—, volatile coefficient of luciferin—, and update coefficient of luciferin—.

Step 2. Update each glowworm .(2.1)Calculate the objective function value of the glowworm’s location—, and then it is turned into the luciferin— according to (1).(2.2)Select the neighbor glowworms in the local-decision domain— and form a neighborhood—.(2.3)Calculate the probability— of glowworm according to (2).(2.4)Select glowworm as the neighbor of glowworm by using the roulette method, and update the location according to (3).(2.5)Let , and a single-dimension— () is selected randomly, then a rand number is generated. If , then ; else .(2.6)Calculate and , and if , the location should be updated or else, keep still.(2.7)Update local-decision domain according to (4).

Step 3. In iteration accumulation, if it reaches , turn to Step 4 or else, turn to Step 2.

Step 4. Output the result.

2.4. Convergence Analysis of Algorithm

The convergence of GSO algorithm is proved mainly through a large number of simulation experiments, and its mathematical theory basis is relatively weak. Krishnanand and Ghose analyzed the convergence of luciferin updating equation in 2006 [20], and they had proved that the maximum luciferin level was bounded due to luciferin decay and the luciferin of glowworms which is located at a peak would converge to the same value according to the deduction of the two theorems.

Theorem 1. Assuming that the luciferin update rule in (1) is used, the luciferin level for any glowworm is bounded above asymptotically as follows: where is the global maximum value of the objective function.

Theorem 2. For all glowworms located at peak location associated with objective function values , with the number of peaks, if the luciferin update rule (1) is used, then increases or decreases monotonically and asymptotically converges to .

In order to show that the convergence of SDGSO algorithm has been improved compared against the standard GSO, and also better than differential evolution (DE) and ABC algorithm, three classical test functions are selected to do the convergence test. The formula of the functions, searching space, and theoretical optimal solution are given in Table 1. In parameters setting, population size is 50, dimension of space is 30, number of iterations is 200, and step size of perturbation is 10−6. SDGSO algorithm and GSO algorithm are tested with the three functions, respectively, for 20 times, and the best value, the median value, the worst value, the mean value, and the standard deviation of the 20 time test are recorded with Best, Median, Worst, Mean, and Std. Among them, Best, Median, and Worst reflect the quality of the solution; Mean shows the precision of the algorithm and reflects the convergence speed of the algorithm; Std reflects the stability and robustness of the algorithm. The results are shown in Table 2.

It can be seen from Table 2 that the precision and stability of the proposed algorithm have been improved obviously compared to DE and ABC algorithm and also better than those of standard GSO algorithm.

In order to illustrate the convergence speed of the improved algorithm more intuitively, convergence curves of GSO and SDGSO algorithms during the solving process of function Griewank and function Rosenbrock are shown in Figure 2. It can be seen that the convergence speed of SDGSO algorithm is faster than that of GSO algorithm.

2.5. Statistical Analyses

In order to statistically analyze the results in Table 2, a nonparametric significance proof known as the Wilcoxon’s rank test [21] has been conducted which allows assessing result differences among two related methods. The analysis is performed considering a 5% significance level. Table 3 reports the value produced by Wilcoxon’s test for pairwise comparison of three groups. Such groups are formed by SDGSO versus DE, SDGSO versus ABC, and SDGSO versus GSO. As a null hypothesis, it is assumed that there is no significant difference between mean values of the two algorithms. The alternative hypothesis considers a significant difference between the two algorithms. All -values reported in the table are less than 0.05 (5% significance level) which is strong evidence against the null hypothesis, indicating that the results of SDGSO algorithm are statistically significant and that it has not occurred by coincidence.

3. Application of SDGSO Algorithm in Block Motion Estimation

In block matching motion estimation, each frame image of video sequences is divided into macro block, and the blocks within the search window of the reference frame will be compared in a certain search pattern. Consequently the best matching block will be found, and the motion vector (MV) can be calculated as shown in Figure 3.

GSO algorithm has strong commonality and a good performance of the low dimensional multimodal function optimization, especially suitable for block matching optimization of 2D images. In this paper, a novel block motion estimation method based on GSO algorithm which combines the good global searching characteristic of GSO algorithm with the single-dimension perturbation strategy and features of the video sequence is proposed. The proposed method not only has improved the accuracy of ME but also has decreased the average number of searching points. Specific implementation process is given as follows.

3.1. Predict the Motion Vector and Initialize the Population

In BMA based on SDGSO algorithm, each macro block is regarded as glowworm, and the center location of the macro block is the location of glowworm. Initial population should be distributed within the search window of the reference frame after selecting the current macro block.

According to the statistics, video sequences have a characteristic called center biased distribution (CBD), and it means that most of the MVs will be concentrated in a central place. Besides, the MV of the current macro block has a similar size, and the same direction with its adjacent MVs, and the MV of the same block in the previous frame; that is to say, there is a strong space-time correlation among motion vectors. Therefore, we use the predictive MV to determine the search center. The average of the left block’s MV, the top block’s MV, the right-top block’s MV and the same block’s MV in the previous frame is regarded as the predictive MV, as shown in Figure 4(a), and then the initial population is distributed around the search center, as shown in Figure 4(b) ().

3.2. Object Function and Luciferin Updating

ME based on BMA is looking for the MV which can get the minimum (or maximum) value of the matching function. Matching function is an important factor of algorithm’s precision accuracy, and the mean average difference (MAD) is chosen as the matching function in this paper which has a small computation and convenient for hardware implementation, and it is defined as follows: where is the pixel value of the current macro block , is the pixel value of the reference block, and and are the horizontal and vertical size of macro blocks.

Luciferin-update equation is associated with matching function, defined as follows:

3.3. Location and Local-Decision Domain Updating

In the process of location updating, size of the step has a great effect on the convergence speed, especially for different motion intensity of video sequences. For example, when the movement is smooth like Akiyo sequence, a small step size like can get the highest search accuracy as shown in Figure 5, and when the movement is intense like Bus sequence, a small step size may slow down the convergence speed, and reduces accuracy. For example, the accuracy is lower when step size of than that of as shown in Figure 5.

Aiming at this problem, the adaptive step size strategy [22] is adopted in this paper, defined as follows: where is called Luciferin factor, is the position of glowworm , is the position of the best glowworm which has the maximum luciferin value, is the maximum distance between the best glowworm and other glowworms, and and are the maximum and minimum step size.

After figuring out the step size, the location of the glowworm should be updated according to (3), and then perform local search of SDPS to determine the final location at the current iteration. Finally, the local-decision domain should be updated according to (4).

3.4. Terminating Strategy

Due to the introduction of SDPS, the calculation of the objective function might be increased, but the speed of finding the optimized solutions is also accelerated. That is to say, the algorithm may find the optimal solution before it achieves the maximum iterations. Therefore, the following strategies are taken to terminate the iteration.(1)Check whether the global optimal location has been changed within the prescribed continuous iterations and if it has not, the iteration should be terminated early.(2)Check whether the maximum iteration is reached and if it is, terminate the iteration.

3.5. Algorithm Steps

For more details, see Algorithm 2.

4. Simulation Results and Performance Evaluation

In the experiments, 5 typical YUV video sequences are chosen according to the different motion intensity: Akiyo (very slow), Foreman (slow), Soccer (fast), Football (fast), and Bus (very fast). The first 100 frames of them are used as the test object, CIF format (352 × 188), 30 frames/second. Under the same test environment, full search (FS), three-step search (TSS), diamond search (DS), adaptive rood pattern search (ARPS), mutation particle swarm search optimization (MPSO) [23] algorithm, glowworm swarm optimization (GSO) algorithm, artificial bee colony (ABC) algorithm, and single-dimension perturbation glowworm swarm optimization (SDGSO) are tested respectively from two aspects of search precision and computation complexity.

Parameters setting: size of macro block is 16 × 16, scope of search window is , . The previous frame is the reference frame. The population , and the maximum iterations . According to the reference value of GSO algorithm and the results of the experiment, set volatile coefficient of luciferin , update coefficient of luciferin , minimum step size , maximum step size , constant factor , modulus , perceive domain , and step size of local search .

4.1. Search Precision

In this paper, the peak signal-to-noise ratio (PSNR) is adopted as the evaluation criteria of search precision. The average PSNR value of the five test video sequences is listed in Table 4. Figure 6 illustrates the value of PSNR with FS of Soccer and Bus sequences, respectively.

It can be seen from Table 4, that the average PSNR value of FS is the largest one, and it means that the search precision of FS is the highest one; DS has a high search precision for the slow motion like Akiyo and Foreman sequences, but for the fast motion sequence, its precision is significantly lower, that is to say, DS only applies to the slow motion; contrary to DS, TSS just applies to the fast motion; ARPS has superior performance compared to other fast motion estimation methods, and it has high search precision and strong applicability; although the average PSNR values of SDGSO and MPSO for slow motion like Akiyo sequence are slightly lower than those of DS and TSS, their overall performances are still higher than those of any other methods because of their high search precision for fast motion sequences as shown in Figure 6; SDGSO has higher search precision compared to MPSO no matter for slow motion or fast motion sequences; in other words, the image restructured by compensating with the motion vectors which are obtained by using SDGSO algorithm will be the most ideal one.

In order to show the good performance of SDGSO algorithm more intuitively, the restructuring images of the fourth frame of Soccer sequence are shown in Figure 7. It can be seen from Figure 7, that the fourth frame images of Soccer restructured by DS, TSS, and ARPS all miss some details, and the images restructured by MPSO and SDGSO algorithm have retained the details of the original image, especially for the latter one, the edge of its objects is almost smooth.

In order to prove that the application of GSO algorithm is better than that of ABC algorithm, the average PSNR value of the two algorithms which take the same population size, the same number of iterations, and the same prediction method of motion vector is listed in Table 5.

It can be seen from the results that the precision of ABC algorithm is worse than that of GSO algorithm no matter for video sequence with slow motion or that with fast motion. That is because the convergence speed of ABC algorithm is slower than that of GSO algorithm, and it cannot converge to the optimal value within the same iteration times. It is also worth noting that the precision of GSO algorithm in Table 5 has not been decreased compared to that of SDGSO algorithm in Table 4. The reason is that ABC and GSO algorithm in Table 5 did not adopt the early termination strategy like SDGSO algorithm, so their searching precisions are relatively high, but it also means that more points need to be calculated; in other words, their high precision is at the cost of high computation complexity, as shown in Table 6.

4.2. Computation Complexity

In block matching motion estimation, the average number of search points for matching a block is used as the evaluation criterion of computation complexity. The average search point of each algorithm is listed in Table 6.

It can be seen in Table 6 that FS method searches every macroblock within the search window, so it needs to search 869 points for one matching block, DS and TSS are based on fixed template, the search point is relatively less, they need to search 20–30 points on average, ARPS is a kind of predictive adaptive algorithm, it has reduced the search points effectively, and the average search points for one matching block are less than 20. MPSO, ABC, GSO, and SDGSO are based on the swarm intelligence algorithm, so their search points are more than those of other fast MEs, but due to the introduction of SDPS, time-space correlation of video sequence and the effective termination strategies, the average search point of SDGSO has been reduced to a certain extent. Besides, the proposed algorithm has a good stability, and its average search point changes little for different test video sequences, so it is supposed to be a robust algorithm.

4.3. SDPS Local Search Strategy

In order to prove that the introduction of SDPS has improved the performance, the application of SDGSO algorithm in ME is compared with that of the adaptive step glowworm swarm optimization (AGSO) algorithm from two aspects of search precision and computation complexity as shown in Tables 7 and 8. The two algorithms have taken the same population size, same times of iterations, same prediction method of motion vector, and the same termination strategies.

It can be seen from Tables 7 and 8 that the search precision of SDGSO which has introduced SPDS is higher than that of AGSO, and the computation complexity of SDGSO is similar to that of AGSO, or even less.

5. Conclusions

In this paper, a novel glowworm swarm optimization algorithm based on single-dimension perturbation search strategy is proposed and applied to block motion estimation (Algorithm 1). The proposed algorithm has overcome the defect of being easy to fall into local optimum by taking advantage of the global optimization ability of GSO and the local optimization ability of SDPS. Besides, it also has reduced the computation complexity by taking the time-space correlation of video sequences and effective terminating strategies. All the experimental results show that SDGSO algorithm has improved both the search accuracy and convergence speed to a certain extant, especially for the block motion estimation of video sequences with fast motion, and good performance is reflected.

Step  1. Set parameters: population size , dimension of space , step size of local search
Step  2. Update the location of all glowworms
Step  3.      for each doing
 (3.1)     {Candidate Solution}
 (3.2)  Randomly choose a dimension to perturb , and generate a rand
      number, rand.
 (3.3)  if then let else let
 (3.4)    Let 
 (3.5)  if then

Step  1. Read the data.
Step  2. Set parameters: , , , , , , .
Step  3. Set the size of block and divide the current frame image into rules of
block.
Step  4. for to do
(4.1) Predict the MV of the current macro block and the center of search window.
(4.2) Generate initial population of glowworms
  (4.3) for each to initializing the luciferin value l(i);
 (4.4)   Set ;
 (4.5) for each ( ) doing
    (4.5.1) for each glowworm i doing
     (4.5.1.1) Form the neighborhood ;
      (4.5.1.2) for each glowworm , computing probability
according
      to the formula (3);
     (4.5.1.3) Select glowworm using ;
     (4.5.1.4) Update glowworm step s with the formula (8);
     (4.5.1.5) Update glowworm position with the formula (2);
     (4.5.1.6) Search the new location by using SDPS;
     (4.5.1.7) Update the luciferin value according to the formula (8);
     (4.5.1.8) Update local-decision domain according to the formula (4);
     (4.5.1.9) if keep unchanged for limt generation then terminating the
iteration;
Step  5. Output MV.

Acknowledgment

This research is supported by the National Science Foundation Council of Guangxi (2012GX NSFAA053227).