Sensing and Intelligent Perception in Robotic ApplicationsView this Special Issue
Research Article | Open Access
Target Centroid Position Estimation of Phase-Path Volume Kalman Filtering
For the problem of easily losing track target when obstacles appear in intelligent robot target tracking, this paper proposes a target tracking algorithm integrating reduced dimension optimal Kalman filtering algorithm based on phase-path volume integral with Camshift algorithm. After analyzing the defects of Camshift algorithm, compare the performance with the SIFT algorithm and Mean Shift algorithm, and Kalman filtering algorithm is used for fusion optimization aiming at the defects. Then aiming at the increasing amount of calculation in integrated algorithm, reduce dimension with the phase-path volume integral instead of the Gaussian integral in Kalman algorithm and reduce the number of sampling points in the filtering process without influencing the operational precision of the original algorithm. Finally set the target centroid position from the Camshift algorithm iteration as the observation value of the improved Kalman filtering algorithm to fix predictive value; thus to make optimal estimation of target centroid position and keep the target tracking so that the robot can understand the environmental scene and react in time correctly according to the changes. The experiments show that the improved algorithm proposed in this paper shows good performance in target tracking with obstructions and reduces the computational complexity of the algorithm through the dimension reduction.
Robot moving target detection and tracking based on computer vision has become a hot spot of research scholars at home and abroad. Its essence is to use effective characteristics of the target to choose effective matching algorithm, have state estimation on the target, and obtain target parameters such as position, velocity, and trajectory and then provide reliable data information for target behavior analysis and understanding . Moving target tracking technology under complex scene needs to achieve sustained and stable tracking for moving object, have certain robustness on the illumination variation of the environment and obstructions of moving target, and improve the running efficiency and accuracy of the algorithm . The tracking technology advice of moving target runs successfully on intelligent robots, realizing the robot formation, scene understanding, and positioning, and it also has the very high economic benefits and social values in real life, such as intelligent automatic monitoring, human body tracking and identification, detection through a red light, and car speeding detection .
In order to solve various problems related to the target tracking, many scholars have done a lot of innovative research work in the field, and, in view of the practical problems of the different areas, many different target tracking algorithms are proposed, which mainly concentrate in the SIFT algorithm, Mean Shift algorithm, and Camshift algorithm. The PCA-SIFT algorithm put forward by Ke and Sukthankar adopts Principal Components Analysis (PCA) to reduce the dimension of SIFT feature vector from 128 to 36 when generating feature vector through the interest points neighborhood gradient direction histogram, which makes the matching speed about three times higher than SIFT algorithm, but also paid the cost of reducing a small amount of matching performance . Mikolajczyk proposes a characteristics description method, GLOH (Gradient Location-Orientation Histogram) with better uniqueness and stronger robustness by extending the SIFT algorithm. GLOH is different from SIFT in the selection of interest points neighborhood and statistical method, and it uses the principal component analysis algorithm to reduce the dimension of feature vector to 128d. Although GLOH has stronger robustness than SIFT, its time complexity is also higher . Liu et al. propose PCA-SIFT algorithm and reduce the time complexity of SIFT algorithm, but also reduce the corner detection accuracy . Collins proposes a way of having the Mean Shift iteration in the defied discrete scale space by adding an extra dimension nucleus, so as to find the method for the best bandwidth . Comaniciu proposes the bandwidth selection problem of kernel function in Mean Shift algorithm, but with complex computation and not weak adaptability . Li uses color fusion template and location prediction to improve the performance of Mean Shift tracking algorithm. On the basis of simple linear fusion for RGB trichromatism of image, select the most prominent fusion feature algorithm in the target features through the target template and the candidate similarity function and set up three target templates accordingly . Guo et al. propose a multiple features fusion tracking method, which combines with color and motion edge character at the same time, and have the probability fusion with the particle filtering method. The combination of Mean Shift algorithm and particle filtering has good robustness for tracking target process in the case of complex environment and changed background, but because of the high algorithm complexity of particle filtering, it is not ideal in real time . Wang et al. propose Camshift target tracking algorithm based on the frame difference and motion estimation, although, to some extent, the proposed algorithm can solve the problem of target occlusion and too fast movement, but when the background is complex and there are other moving targets, it cannot track well. At the same time, the algorithm cannot solve the problem of similar target jamming . Qin combines the space histogram and the Camshift, but the algorithm has some shortcomings; for example, background information in the operation area may also be regarded as target information in statistical histogram, which reduces the precision of the histogram, influences on target tracking. At the same time, the algorithm is used in H layer of HSV space and it is hard to extract the targets when the brightness of the target and background are similar . Huang combined frame difference method to realize the Camshift automatic tracking, but because frame difference method only extracts the edge of the part outline to make the holes inside the target, it tends to cause the incomplete or containing background in target extracting or reduce tracking quality for containing the other colors in the color features . Li and others improve the interested area extraction in Camshift algorithm with Canny filtering, making the acquisition of target color histogram resist the influence of noise, while the method is not effective when the background profile is taken into the detection range . Cai improves Camshift tracking performance with three-frame difference method, which makes the extraction of the target more accurate, but still cannot get rid of the inherent defect of frame difference method .
This paper proposes a target tracking algorithm integrating reduced dimension optimal Kalman filtering algorithm based on phase-path volume integral with Camshift algorithm, improving the existed efficiency in target tracking aiming at Camshift algorithm.
2. Performance Analysis of Target Tracking Algorithm
The current common moving object tracking algorithms include SIFT algorithm, Mean Shift algorithm, and Camshift algorithm. This paper first has a performance analysis on the three kinds of algorithms.
2.1. Defect Analysis of SIFT Algorithm
SIFT algorithm extracts the target key points from the image, and these feature points have invariance in the image scaling, rotation, some degree of light intensity, and camera angle changing.
The first step of key points detection is on the insensitive pixel location for the changes of image scales in different scales. The scale space of the image is defined as a function of , which is from the input image and convolution of Gaussian kernel function with different scales:in which is a convolution operation about and . Consider
In order to detect the key position effectively in image scale space, the differential Gaussian function is used here, which can be composed of image convolution results in two adjacent scales by subtracting
Then set the gotten extreme value point as alternative key point, using the Taylor expansion of the scale space function :in which and its expansion item calculate in the sampling point and is the compensation amount of the point. When the expansion item of is 0, the corresponding is the extreme point, and the position is decided by the following formula:
In the specific calculation, the difference value between adjacent sampling points is used as the approximate scale space function and its expansion. If the amount of compensation in various scales is greater than 0.5, then the distance between the current sample point and the extreme value point is regarded as not the shortest. Next exclude the current sample point and continue the same calculation for the other sample points. Finally get the approximate estimation of extreme value point location by combining with its corresponding sampling points.
By setting the threshold value, abandon the extreme value points whose is less than the threshold value. A typical threshold size can be set as 0.05.
It can be seen from the above analysis that SIFT algorithm relies on extracting features in great quantities; thus it achieves precise matching with information redundancy strategy and, at the same time, also increases the amount of calculation of the algorithm itself.
2.2. Defect Analysis of the Mean Shift Algorithm
Mean Shift algorithm is a kind of estimation technique with smooth and density estimation, and algorithm convergence is the heart of determining the precision of the target tracking. The convergence of Mean Shift and the increasing of the value is equivalent to the proven proposition: if kernel function is defined by a descending concave function , then sequences and are decreasing convergence.
As is limited, and sequence is bounded, according to the definition of sequence ,
According to the definition of concave function, for all , , they have
As , formula (8) can be written as
Substituting the above formula into (7), we can get
We can get
As is monotone decreasing in , so is a positive real number, for if only , is a nonnegative real number. This shows that is an increasing sequence. Since it is bounded, it is a convergent sequence.
Then accumulate items of (11); we can get
means the minimum of the in items of the above formula. It shows that is Cauchy series, and is also convergent.
As can be seen from the above analysis, the Mean Shift algorithm will fail in tracking when the target scale changes due to the fact that the window width remains the same in the target tracking process, and when the target speed is high, the tracking effect is not good.
2.3. Defect Analysis of Camshift Algorithm
Extend Mean Shift algorithm to the continuous image sequence, which generally refers to the image sequence; thus it forms the Camshift algorithm. The basic idea of Camshift algorithm is to have Mean Shift algorithm for all the frames of the image; then set the center and size of the search window on the previous frame as the initial value of the search window on the next frame, and the target tracking can be achieved with such iteration.
Camshift algorithm tracks target with elliptical lock. The size and the deflection angle of the tracking target are shown in Figure 1.
Define the secondary moment as follows:
Then long axis and short axis of target ellipse area, respectively, are
In the formula, the values of , , are obtained with the following formula:
The target angle of the direction is defined as the included angel less than between long axis and the horizontal plane. Consider
Camshift algorithm is to set the result of the previous frame as the initial value of the next search window frame; then find optimal target location in the field of the center coordinates. If the target moves too fast, which makes the distance between the current frame and the previous frame center larger, the previous location is still used in the current frame search as the initial position, which will lead to increasing times in searching target, and even failure in tracking target.
2.4. Performance Analysis on the Three Algorithms
Use ten different image sequences to test the target tracking of the above three algorithms, and extract 100 observation sequences from each image sequence to check the accuracy of target tracking. The algorithm testing results of five image sequences without obstructions are shown in the Figure 2. And the testing results of the other five image sequences with obstructions are shown in Figure 3.
After analyzing the results of the experiment, Camshift algorithm has better performance in terms of target tracking. So this paper will apply Camshift algorithm to target tracking and improve the efficiency of Camshift algorithm.
3. Target Centroid Position Estimation Optimization of Camshift Algorithm
3.1. Reduced Dimension Optimization of Kalman Filtering Algorithm
This paper adopts the integration of Camshift algorithm and Kalman algorithm to improve the defects of Camshift algorithm but considers that it improves the complexity of the algorithm and greatly increases the amount of calculation after combining Kalman algorithm, so the optimization in calculation amount of Kalman algorithm comes first.
The sampling point number of Kalman algorithm increases with higher dimension in dealing with nonlinear system, and filtering calculation also increases, while the Gaussian integral has the greatest impact on the calculation of Kalman algorithm, so phase-path volume integral is used instead of Gaussian integral in Kalman algorithm to reduce dimension. In the case of no loss of filtering precision, it reduces the number of sampling points and the amount of calculation in the process of filtering.
Phase-path volume integral is an approximate integral after nonlinear function passing to a series of point set and weighted sum, but the requirements for the point set are as follows.
If , change the order and the symbols of the elements in to get ; there are still , and the weights of the two factors are equal, and is called the holohedral symmetry.
For an element in holohedral symmetry area of phase-path volume integral, if , , then can be defined as the generator of the holohedral symmetry area in dimensional real number area. At the same time, define as the same collection with . Set the Gaussian integral as
If constant is not considered, the integral can transform by integral conversion into
Among them, is the unit volume of dimension, and is the element on ; at this point, the integral of formula (18) is converted into a volume integral and a phase-path integral:
The volume integral in formula (20) can be approximate with volume rules. It can be known from the holohedral symmetry rules of the volume that for every monomial in , in which is the variable order; when is odd integer, the integral is 0. So if we want to be accurate to three monomial, namely, approximate to the integral with the phase-path volume rules, and as long as the two points can be accurately obtained with the chosen volume point and the corresponding weights, only the conditions and , that is, and , need to be considered. The formula is as follows:in which means the unit volume of dimension, and .
For the phase-path integral type, set , and it can be transformed through the integral conversion into
According to the rules of phase-path volume integral, when or , that is, when or , the accurate integral type can be obtained. At this time, the selected integral points and weights, respectively, are
Then, use the phase-path volume integral instead of Gaussian integral in Kalman algorithm. The nonlinear system of Kalman algorithm can be shown in the following form:in which is the previous element of ; that is, .
If statistical properties of at the time are known, then the filtering function of Kalman algorithm is as follows:in which and are the Gaussian integrals about . Consider
In the standard Kalman algorithm, the number of sampling points on state is , for is dimension vector; if the phase-path volume is used to approximate formula (33), then only sampling points of are needed. The formulas are as follows:
3.2. Camshift Algorithm Based on Kalman Prediction Estimation
The basic idea of Camshift algorithm based on Kalman prediction estimation is that set the optimal estimation of target centroid in current frame as the initial position of the Camshift algorithm search window in the next frame and then revise the predicted value with the centroid result of Camshift algorithm iteration as the observation value of the improved Kalman filtering algorithm, so as to obtain the optimal estimation of target centroid position of the next frame. The forecast of target centroid position can solve the problem of quick target motion effectively, making sure the search window of Camshift algorithm can keep up with the movement of the target, improving the effect of tracking.
Definition 1. The state vector of Kalman filtering algorithm is , in which and represent the target centroid position, and , respectively, represent the displacement speed of the target in the direction of and . Define observation vector as , in which and represent the target centroid position obtained with Camshift algorithm. Due to the fact that the time interval between two frames in image sequence is short, we regard it as uniform motion in unit time interval and then define the transition matrix as follows:in which is the time interval between and . From the system state and observation state, the observation matrix is as follows:The estimation on the centroid position of moving targets using improved Kalman filtering algorithm in Camshift tracking algorithm can be roughly divided into three stages, which are as follows.
(1) Initialization of the Kalman Filtering Algorithm. Initialize improved Kalman filtering algorithm, initialize with the target centroid position and moving speed obtained by the moving target detection algorithm, and record the current time.
(2) Centroid Position Estimation. Calculate the time interval with the previous frame image when processing the current frame image and substitute it in the transition matrix . Use the iterative operation by taking the location of as the center of the Camshift algorithm search window and then substitute the searched final location into the observed value to correct predicted value.
(3) Location Update. Substitute the state vector and the observation vector into the recursive formula of improved Kalman filtering algorithm, get the optimal target centroid position estimation of target area centroid in the current frame, and record the moment of the current frame image and target centroid position.
4. Algorithm Performance Test
First take performance test to the improved Kalman filtering algorithm in order to verify the effectiveness of the improved algorithm proposed in this paper, having 100 Monte Carlo simulations under the same condition with the standard Kalman filtering algorithm and reduced dimension Kalman filtering algorithm (IM-Kalman). The applied software is Matlab, and the computer processor is Core I7-3610 with the main frequency of 2.3 GHz, memory of 4 G. The mean absolute error curves of two filtering algorithms for state estimation are shown in Figures 4~6.
When processing the single frame image, the accuracy of state estimation of standard particle filter (PF) algorithm, standard Kalman filter algorithm and IM-Kalman filter algorithm is contrasted, respectively, at the particle number of 50, 100, and 150, and the results are shown in Tables 1 and 2.
It can be seen from Table 1 that when the particle number is 50, the time used in single frame image processing of the three algorithms has no big differences, while the time used of the improved Kalman filtering algorithm proposed in this paper is slightly less than that of the other two algorithms. But when the particle number increases to 100, the processing speed of PF algorithm and the standard Kalman filtering algorithm is significantly slower, while the speed of IM-Kalman filtering algorithm proposed in this paper changes slightly. When the particle number increases to 150, the processing speed of IM-Kalman filtering algorithm proposed in this paper is much higher than the PF algorithm and the standard Kalman filtering algorithm.
As can be seen from Table 2, when the particle number is 50, there has been a certain gap among the state estimation accuracy of three algorithms, and the accuracy of IM-Kalman filtering algorithm proposed in this paper is much higher than the other two algorithms. When particles are increased to 100, the precision of PF algorithm and the standard Kalman filtering algorithm has been reduced slightly, but there is no significant reduction in the IM-Kalman filter algorithm proposed in this paper. When the particle number increases to 150, the estimation accuracy of the PF algorithm and the standard Kalman filtering algorithm has been less than 1, while the estimation accuracy of IM-Kalman filter algorithm reduces a little.
The simulation results show that IM-Kalman algorithm has better estimation precision and uses less time. Then the standard Camshift algorithm and the improved Camshift algorithm are used for target tracking the four 100 frames video sequences, which are human motion video sequences without obstructions (Test 1), human motion video sequences with obstructions (Test 2), vehicle motion video sequences without obstructions (Test 3), and vehicle motion video sequences with obstructions (Test 4). Results are shown as follows.
(1) The Human Body Target Tracking. First have the human body target tracking with obstructions and without them. Figure 7 shows target tracking result of the standard Camshift algorithm in the absence of any obstructions, and Figure 8 shows the result of improved Camshift algorithm in the same case. Figure 9 is the target tracking result of standard Camshift algorithm with obstructions, while Figure 10 is the result of improved Camshift algorithm in the same case.
It can be seen from the above experimental results that the improved Camshift algorithm put forward in this paper has better effect than the standard Camshift algorithm in the human body tracking with obstructions.
(2) Vehicle Target Tracking. Then have the vehicle target tracking with obstructions and without them. Figure 11 shows target tracking result of the standard Camshift algorithm in the absence of any obstructions, and Figure 12 shows the result of improved Camshift algorithm in the same case. Figure 13 is the target tracking result of standard Camshift algorithm with obstructions, while Figure 14 is the result of improved Camshift algorithm in the same case.
In the above experiment, the results of processing time on four video sequences with two algorithms are shown in Table 3.
It can be seen from the above experimental results that the improved Camshift algorithm proposed in this paper has better effect than the standard Camshift algorithm in the vehicle tracking with obstructions, improving the efficiency and accuracy of the intelligent robot target tracking algorithm under complex environment. From the comparison results of the processing time of two algorithms in four video sequences, the improved Camshift algorithm proposed in this paper uses relatively less time and is with excellent performance.
This paper proposes a target tracking algorithm integrating reduced dimension optimal Kalman filtering algorithm based on phase-path volume integral with Camshift algorithm, reduces dimension of Kalman filtering algorithm, fixes the predictive values with the target centroid position as the observation value of the improved Kalman filtering algorithm, and thus takes the optimal estimation for target centroid position. Under the condition of the target with obstructions, it can detect the target centroid position accurately, solve the problem of transient target occlusion, improve the running speed of intelligent robot system, and have higher efficiency and robustness in tracking process.
Conflict of Interests
The author declares that there is no conflict of interests regarding the publication of this paper.
This work was supported by the Department of Science and Technology of Zhejiang Province (Grant no. 2014C31065), Zhejiang Education Scientific Planning Research (Grant no. 2014SCG104), and Zhejiang Shuren University Young Academic Team Project.
- Y. Xie, “Tracking objects based on partial least squares analysis using particle filtering with dual models,” Control and Decision, vol. 29, no. 8, pp. 1372–1378, 2014.
- G. Hua, “A moving objects tracking system of robot based on modified optical flow algorithm,” Control Engineering of China, vol. 21, no. 5, pp. 634–637, 2014.
- L.-J. Wang, S.-M. Jia, X.-Z. Li, and S. Wang, “Person following of binocular robot by extracting multiple features,” Control and Decision, vol. 28, no. 10, pp. 1568–1572, 2013.
- Y. Ke and R. Sukthankar, “Target recognition algorithm based on PCA-SIFT,” Tv Engineering, vol. 37, no. 15, pp. 30–32, 2013.
- S. Mikolajczyk, “GPU-based real time image registration with variant SIFT,” Computer Science, vol. 38, no. 3, pp. 300–303, 2011.
- Z. Liu, B. Xing, and Y. Chen, “An efficient parallel PCA-SIFT algorithm for multi-core processor,” Journal of National University of Defense Technology, vol. 34, no. 4, pp. 103–107, 2012.
- R. T. Collins, “Mean shift based FCM image segmentation algorithm,” Control and Decision, vol. 29, no. 6, pp. 1130–1134, 2014.
- D. Comaniciu, “Improved mean shift tracking algorithm based on complicated feature fusion,” Control and Decision, vol. 29, no. 7, pp. 1297–1300, 2014.
- J. Li, “Mean shift target tracking based on color and LBP features,” Journal of Hefei University of Technology (Natural Science), vol. 37, no. 5, pp. 578–581, 2014.
- J. Guo, X. He, J. Yang, Z. Wei, and J. Gong, “Infrared target tracking based on template adaptive Mean Shift,” Infrared and Laser Engineering, vol. 43, no. 4, pp. 1087–1093, 2014.
- Z. Wang, L. Wu, W. Cai, and J. Chen, “A modified algorithm of video target tracking based on camshift,” Automation & Information Engineering, vol. 35, no. 1, pp. 27–31, 2014.
- Y. Qin, “Improved Camshift objects tracking algorithm based on three-dimensional histogram,” Modern Electronic Technique, vol. 37, no. 2, pp. 29–33, 2014.
- Y. Huang, “Eye tracking method on improved CamShift algorithm,” Application Research of Computers, vol. 31, no. 4, pp. 1220–1224, 2014.
- D. Li, J. Tian, L.-Q. Xiao, and J.-P. Sun, “Adaptive multi-feature template video target tracking based on Camshift algorithm,” Journal of the China Coal Society, vol. 38, no. 7, pp. 1299–1304, 2013.
- C. Cai, “CAMShift face tracking with adaptive MB-LBP pre-filter,” Signal Processing, vol. 29, no. 11, pp. 1540–1546, 2013.
Copyright © 2016 Fengjun Hu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.