Abstract
Ultrasonic image sequence of the soft tissue is widely used in disease diagnosis; however, the speckle noises usually influenced the image quality. These images usually have a low signal-to-noise ratio presentation. The phenomenon gives rise to traditional motion estimation algorithms that are not suitable to measure the motion vectors. In this paper, a new motion estimation algorithm is developed for assessing the velocity field of soft tissue in a sequence of ultrasonic B-mode images. The proposed iterative firefly algorithm (IFA) searches for few candidate points to obtain the optimal motion vector, and then compares it to the traditional iterative full search algorithm (IFSA) via a series of experiments of in vivo ultrasonic image sequences. The experimental results show that the IFA can assess the vector with better efficiency and almost equal estimation quality compared to the traditional IFSA method.
1. Introduction
The motion estimation not only is an important component of the block-based video compression but also is often applied in disease diagnosis for the image sequences of the ultrasound or the magnetic resonance images. The ultrasonic examination is widely used to present the soft tissues such as heart or muscle; however it is usually influenced by speckle noises and the temporal decorrelation of the speckle patterns [1]. The popular optical flow algorithm [2] is based on the assumption that the intensity of material point of the tissue in a complete image sequence is always invariant. This assumption is generally called intensity invariant constraint; however, there exist the speckle noises and the tissue deformations usually destroy the intensity constraints and therefore cause the difficulty of motion estimation.
Another approach for motion estimation problem is the block-matching algorithm [3–5]; it measures the motion vector by evaluating the similarity criterion, such as the mean square error (MSE), mean absolute error (MAE), and sum of absolute differences (SAD) between the corresponding blocks of two consecutive images. The block-matching algorithm is to find a candidate block within a search region of the corresponding image that best matches the current block in the current image. The displacement between these two matching blocks is called a motion vector (MV). The full search algorithm (FSA) [6, 7] is the simplest block-matching algorithm that can search in burst-force for the optimal estimation solution with minimal matching error as it checks all candidate search points at a time. However, it is extremely computational expensive at the searching process, and, therefore, its usage is seriously limited in the medical image sequences. The effective reduction of the number of search points can accelerate the motion estimation, which is based on the assumption that block distortion is monotonically decreasing. These algorithms are proposed to alleviate the high computational complexity of full search, including the three- or four-step search [8, 9] and the gradient descent search [10]. However, the ultrasonic images are usually noisy and with low signal-to-noise ratio due to the speckle noises and the decorrelation of the speckle patterns; as a result the monotonically decreasing assumption does not hold such that the usage of these algorithms always traps into local optima. In order to tackle this problem, the traditional motion estimation algorithms are revised into an iterative algorithm together with a specific smoothness constraint; however, these changes need a great amount of computation to estimate optimal motion vectors when we use the iterative full search algorithm (IFSA).
Recently, the algorithms of bioinspired computing (BIO) such as the honey bee mating algorithm [11] and shuffled frog-leaping algorithm [12] have been used to efficiently search for the global optimum in the complex optimization problems. Up to the present, there are few of BIO methods that addressed the problems of motion estimation in the ultrasonic image sequence. Among them, the firefly algorithm is another algorithm that had been used in the image thresholding and vector quantitation [13, 14]. In this paper, we present an iterative algorithm that builds in the firefly algorithm for motion estimation. The proposed algorithm is introduced in Section 2. In Section 3, we demonstrate the experimental results. The conclusion and final remarks are given in Section 4.
2. Material and Methods
The problem of estimating motion field from image sequence has been studied extensively. The block-matching algorithm is the most popular because of its simplicity. Section 2.1 will briefly describe the block-matching algorithm. Section 2.2 introduces the firefly algorithm. In order to handle the problem of speckle noises, we designed an iterative firefly algorithm (IFA) together with a smooth constraint to estimate the motor vectors, described in Section 2.3.
2.1. The Block-Matching Algorithm
Because of its simplicity, the block-matching is a widely used algorithm in motion estimation. In matching procedures, the estimated image block of the processing frame will correspond to the best matching location within the predefined search window of the reference frame, as shown in Figure 1. In general, the size of estimated block, ispixels and the size of the corresponding search window is . These two windows are centered at the same pointin the two consecutive image frames (and). The full search searches for all possible locations within the search window by evaluating some matching criteria and selected one locationof the corresponding block. The relative displacement of the two locations of bothand is defined as the motion vector,,, .
(a)
(b)
The estimation of the motion vector is to find the optimal one such that its corresponding mean square error (MSE) is the smallest. Equation (1) defines the MSE measure: In other words the optimal motion vector satisfies the following equation:
2.2. Firefly Algorithm
The firefly algorithm is a new bioinspired computing approach for stochastic optimization, in which the search mechanism is simulated by social behavior of fireflies and the phenomenon of bioluminescent communication. There are two important issues in the firefly algorithm that are the variation of light intensity and formulation of attractiveness. Yang [15] simplifies that the attractiveness of a firefly is determined by its brightness which in turn is associated with the specific fitness function. The attractiveness is proportional to their brightness. Every member of the firefly swarm is characterized by its bright in which one can directly measure the corresponding fitness value.
Furthermore, there are three idealized rules.Regardless of their sex, any one firefly will be attracted to other fireflies.Attractiveness is proportional to their brightness, so for any two flashing fireflies, the less bright one will move to the brighter one. If there is brighter one than the particular firefly, it will move randomly.Brightness of a firefly is affected or determined by the landscape of the given fitness function; in other words, the brightnessof a firefly can be defined as .
More precisely, the attractiveness between firefliesandis defined as any monotonically decreasing function of their distance by the following equations:whereis the sum of initial assigned brightness of these two fireflies, is the light absorption coefficient, andindicates the index of the dimension of the candidate solution (i.e., firefly).
The movement of a fireflythat is attracted to another more attractive fireflyis determined by the following equations: If there is no brighter one than a particular firefly,, it will move randomly according to the following equation:where andare random numbers obtained from the uniform distribution.
2.3. The Proposed Motion Estimation Algorithm Using the Firefly Algorithm
The firefly algorithm usually defines a fitness function to compute the brightness of fireflies. Traditionally, the motion estimation algorithms use a similarity measure as their fitness function such as the mean square error (MSE) [3, 4]. However, the fact that speckle noises are exhibited in ultrasonic image sequence so that the traditional similarity criterion is not enough to accurately estimate motion vectors. To further decrease the sensitivity to speckle noises, a smoothness constraint is adopted to force continuity among neighbors motion vectors. Therefore, the used fitness function of the proposed algorithm consists of two terms of the similarity measure and the smoothness constraint. The widely used similarity measure is the mean square error (MSE), when the best motion vector is found; its corresponding MSE is the minima. The smoothness constraint is necessary to refer to its neighbors pixel’s motion vector to estimate current motion vector, and therefore it must be iterative. More practically, the calculation of motion vector at iterative number must consider the motion vectors of its neighbors at iterative number. The detail of the designed smoothness constraint is described as follows.
Suppose that the estimating motion vector of current blockwith center pixelat iterative number be the , and then the estimated motion vectors of block with the center of pixel at iterative number are . The is the eight neighbors of point . The smooth constraints (SC) at the iteration are described in the following equation:
The fitness function is defined as (7); it is used as search criteria for estimating motion vector in this paper: The proposed algorithm is described as follows.
Step 1 (setup the parameters of proposed system). This step assigns the parameter of the number fireflies, the maximum iteration numberand the light absorption coefficient, and the smooth parameter (In addition, according to (6), the estimation of the motion vector of block needs these initial motion vectors of neighbor. Therefore, all of the initial motion vectors of pixels of estimating block are randomly given ranging from to .
Step 2 (assign the initials of each firefly). For each estimating block , we assigned fireflies (in general, ) and each firefly is assigned an initial motion vector ranging from and ; that is, denotes the iterative number and is initially assigned to be 0 and the initial brightness of each firefly is assigned by its corresponding fitness.
Step 3 (update all candidate solutions). The mechanism of updating of candidate solution is generally stochastic; more practically, the IFA sequential picked up current th firefly solution and randomly selects its corresponding th firefly solution. The two motion vectors are denoted by and If the fitness value of is less than the fitness of , will update toward according to the following equations:where is the sum of initial assigned brightness (fitness value) of these two fireflies and γ is the light absorption coefficient: All of the candidate solutions are updated according to the procedures of (9) and (10), and then the best one is calculated for later process.
If theth firefly is the best one among all fireflies, this firefly will perform a random walk to renew its solution by using (11). If the new one has better fitness than the original one, it will replace it with the best solution; otherwise it is discarded. Finally, the best solution will be recorded by:
Step 4 (iterative execution and resulting vector output). The iteration number adds by 1: Ifreaches the maximum iteration number , the algorithm outputs the as the resulting motion vector; else go to Step 3.
3. Experimental Results
3.1. Execution Environment
All programs are programed by using Visual C++ programming language on a personal computer 2.4 GHz CPU, 1 G RAM running window XP system. All experiments were conducted in parameters of light absorption coefficient,. The size of initial firefly is assigned to be 5 and the maximum iteration number. The sizes of estimated image region, estimated block, and search window are, , and (i.e., ).
Because of the substantial computational power of modern ultrasound systems, it is potential to evaluate muscle motion in real time [1]. The forearm extensors and extensor tendon are at the back of the forearm that typically cause pain at the outer aspect of the elbow. These muscles and tendon act to extend the wrist and fingers to bend the wrist backwards and straighten the fingers. In experiments, images were obtained by scanning the extensor tendon in longitudinal cross section. The images were taken from the tendon portions when the subject’s wrist moved extension to flexion. Figure 2 shows an ultrasonic extensor tendon image sequence, whose frame rate is 39 frames of one second.
3.2. The Selection of Smooth Parameter λ
The setting of smoothness parameter may influence the estimation quality of motion estimation algorithm; we explored the different smoothness parameters in experiments. In general, the estimation quality is characterized by the peak signal-to-noise (PSNR) value; more practically, in PSNR the signal is the original data frames; meanwhile, the noise is the error introduced by the calculated motion vectors. The definition of PSNR is expressed as
The MSE is the mean square error between the original frames and those compensated by the estimated motion vectors. Figure 3 showed an illustration of results of PSNR versus the iteration number under different smoothness parameters. Form this figure, we found that the setting of the smoothness parameter was not significant to the final results of IFA and IFSA algorithms, and further the IFSA is in need of 20 iterations to converge but the iterative firefly algorithm requires 200 iterations. As a result, the smoothness parameter of all follow-up experiments was assigned to be 1.0.
3.3. Performance Evaluation
For comparisons with two different algorithms, two relevant performance indices were considered: the estimation quality and the search efficiency.
First the IFSA and IFA algorithms were compared in terms of estimation quality. The widely used index is the PSNR defined as (12); in addition to PSNR, an alternative index called PSNR degradation ratio () [5] is used in the experiments, which expresses the level of mismatch between the PSNRs of IFSA and IFA methods. The is defined as
Figure 4 shows the 10 selected regions of interest (ROI) with size ofpixels in an ultrasonic extensor tendon image sequence; those are used to estimate their motion vector by using the IFA and IFSA. Tables 1 and 2 show the PSNR values and computational times under running 200 iterations, considering the ten ROIs of Figure 4. Tables 3 and 4 are the results of PSNR versus computational time under 300 iterations. As it can be seen from these tables the quality results of both the IFA and IFSA have a limited difference. The corresponding PSNR degradation ratios are only 0.02105% (200 iterations) and 0.002% (300 iterations). The little differences imply that the usage of IFA is an alterative to calculate the accurate motion vectors. Figure 5 shows the resulting motion vectors of the first ROI using the IFSA and IFA ran with different iterations. In this figure, the interval time between current ROI and reference ROI is second.
(a) An image of ultrasonic extensor tendon
(b) Current ROI (defined in (a))
(c) Reference ROI (next frame of (a))
(d) IFSA with 20 iterations
(e) IFA with 50 iterations
(f) IFA with 100 iterations
(g) IFA with 150 iterations
(h) IFA with 200 iterations
(i) IFA with 300 iterations
In addition, the computational times of the IFA and IFSA methods serve as an index for measuring the search efficiency. The index call computation saving ratio (CSR) [5] is defined for comparison in the experiments. It expresses the level of computation saving ratio between the computational times of the IFA and IFSA. The CSR is defined as
Tables 1 and 3 show the computational time of estimating of the 10 ROIs using both IFSA and IFA. From the two tables we found that the average computational time of IFA is 8.20 seconds (200 iterations) and 12.301 seconds (300 iterations), whereas the IFSA needs 30.785 seconds (20 iterations). Tables 5 and 6 show that the average CSRs of IFA are 73.4163% (200 iterations) and 60.1631% (300 iterations); in other words, IFA can save in excess of 60% computational cost.
4. Conclusion
Based on the results of the present experiments of ultrasonic extensor tendon image sequence, the following conclusions can be emphasized.(1)In spite of the fact that the IFSA is a widely used algorithm to calculate the motion vector, it needs a great amount of computational time to calculate motion vectors, especially in the medical image sequences.(2)The IFA method is a stochastic algorithm that searches for optimal solution according to the swarm behavior of fireflies. This mechanism can effectively tackle the influences of speckle noises.(3)Experimental results revealed that the estimation qualities of the IFA and IFSA are nearly equal, and the IFA can save in excess of 60% computational cost. It implies that the proposed IFA is a more powerful algorithm in motion estimation of the application of ultrasonic image sequence.(4)The IFA pioneered in an application of bioinspired algorithm in estimating the motion field of ultrasonic image sequence. Maybe we can further investigate other bioinspired algorithms in the same application motion estimation. In addition, the IFA is a potential to be extensively applied in other motion researches such as motion vector composition [16, 17].
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
The work is supported by the Ministry of Science and Technology, Taiwan, under Grant nos. NSC 102-2221-E-251-001 and MOST 103-2221-E-153-011.