Advances in Variational and Partial Differential EquationBased Models for Image Processing and Computer Vision
View this Special IssueResearch Article  Open Access
Bin Zhu, Lianfang Tian, Qiliang Du, Qiuxia Wu, Lixin Shi, "An Improved FractionalOrder Optical Flow Model for Motion Estimation", Mathematical Problems in Engineering, vol. 2018, Article ID 6278719, 6 pages, 2018. https://doi.org/10.1155/2018/6278719
An Improved FractionalOrder Optical Flow Model for Motion Estimation
Abstract
The Horn and Schunck (HS) optical flow model cannot preserve discontinuity of motion estimation and has low accuracy especially for the image sequence, which includes complex texture. To address this problem, an improved fractionalorder optical flow model is proposed. In particular, the fractionalorder Taylor series expansion is applied in the brightness constraint equation of the HS model. The fractionalorder flow field derivative is also used in the smoothing constraint equation. The EulerLagrange equation is utilized for the minimization of the energy function of the fractionalorder optical flow model. Twodimensional fractional differential masks are proposed and applied to the calculation of the model simplification. Considering the spatiotemporal memory property of fractionalorder, the algorithm preserves the edge discontinuity of the optical flow field while improving the accuracy of the estimation of the dense optical flow field. Experiments on Middlebury datasets demonstrate the predominance of our proposed algorithm.
1. Introduction
Since the optical flow was proposed by Gibson in 1950, many scholars have done research on it. In these studies, the HS optical flow model proposed by Horn and Schunck [1] in 1981 achieved high reliability and accuracy and accordingly began to attract more and more scholarly attention. But the accuracy of the motion estimation with the HS optical flow model would be greatly reduced in the image sequence, which would include complex texture and nonrigid motion; meanwhile the discontinuity of motion estimation is difficult to retain.
Many scholars have devoted themselves to the modification or improvement of HS model to increase the accuracy [2], deal with large displacement motion [3], solve the occlusion problem [4], and track nonrigid motion targets [5, 6]. All these models are based on firstorder differentiation techniques.
Lv et al. [7] added a nonlinear fourthorder diffusive term to the EulerLagrange equations of the variational TV model, and the improved model can preserve sharp jump discontinuities, but it would deteriorate when too much noise exists. A highorder total variation minimization method for image deblurring and denoising was proposed by Chan et al. [8]; the model can filter out noise while the edge discontinuity was retained. But details of the edge were neglected.
Because of its ability to preserve the texture details of the smooth region while highlighting the image edge feature and its spatiotemporal memory of the target point neighborhood, a fractional differential is applied to many image processing fields [9], such as image denoising [10], image enhancement [11], and motion estimation [12–16]. In [12, 13], the fractionalorder smoothing constraint equation was used in the HS optical flow model to preserve the discontinuity of motion estimation, but it does not consider the correlation of the pixel intensity. Although the edge discontinuity of the image can be preserved to a certain extent, the accuracy of the optical flow field estimation is not high. In order to further enhance the accuracy of motion estimation, an improved fractionalorder optical flow model is proposed in this paper.
The main characteristics of the present paper can be summarized as follows:(1)Fractionalorder was first used in a brightness constraint equation of the HS optical flow model.(2)A twodimensional fractionalorder differential mask, which was used to compute dual fractionalorder differentials, was first proposed and applied to the simplification of the improved fractionalorder optical flow model.(3)Combined with the EulerLagrange equation and twodimensional fractionalorder differential masks, the iteration formula of the optical flow field estimation of the improved fractionalorder optical flow model was constructed, which is different from any of the former counterparts.
2. The Construction of TwoDimensional FractionalOrder Differential Masks
The mathematical preliminaries of the fractionalorder theory used in this paper can be found in [17, 18].
In this chapter, six twodimensional fractional differential masks were presented. There are , , , , , , where is the order of fractional differentials.
We define as the twodimensional fractional differential mask of dual derivatives on the axis. The twodimensional fractional derivatives of order of a discrete image brightness function on the axis can be defined as
is the mask size, when :
Similarly, where , when :Also,where , , . When ,
What is more, we can get , .
Normalization should be done in all the mask items. More details about the expansion of dual fractionalorder derivatives can be found in [12].
3. Improved FractionalOrder Optical Flow Model
3.1. The FractionalOrder Optical Flow Model
We substitute the brightness constraint equation of the HS model with the fractionalorder Taylor series expansion and get the fractionalorder brightness constraint equation aswhere is the abbreviation of .
Combined with (2.19) and (2.20) in [12], we can get the fractionalorder optical flow model in this paper:
3.2. Numerical Algorithm
The EulerLagrange equation [19] was applied in this paper to minimize energy function .
After simplification, we get the equation as follows: where
After a series of computations, the iteration formula for the proposed fractionalorder optical flow model can finally be written as where , represented the optical flow vector in the neighborhood of point , and
After repeated iteration utilizing the GaussSeidel iteration method, the accurate optical flow field can be achieved, and the convergence of the algorithm can be demonstrated by the method mentioned in [13].
4. Experimental Results and Analysis
We have conducted our experiments from the following three aspects: the comparison of the performance of previously tested optical flow algorithms; the influence of the order of fractionalorder differentials for the algorithm; and the influence of the size of the dual fractional differential mask window for the algorithm. The Middlebury datasets [20] are chosen for evaluating our proposed algorithm for its characteristics of having different structures and attributes, like textures, blurred images, discontinuities in motion, shadows, nonrigid motion, and so on.
The angular error (AE) and flow endpoint error (EE) are selected for evaluation. As we are concerned with the performance of the algorithm in the whole image, the AE/EE in the entire image (AVAE/AVEE) are used to visually evaluate our proposed optical model.
For convenient description, we call the HS optical flow model the HS model, the model in [12] the FOVOF model, and our proposed model the DFOVOF model. The mask window size is .
Figure 1 shows the optical flow field of the HS model, the FOVOF model, and the DFOVOF model in different image sequences. In the DFOVOF model, , , . Among them, different colors represent different movement directions and different color depth represents different speed of movement. The deeper the color, the higher the speed.
(a) Yosemite
(b) Grove
(c) Urban
(d) Army
The first row in Figure 1 shows the acquired optical flow field produced by the traditional HS model. It can be seen that the edge of the image discontinuity is not well reserved and the details of the image texture are neglected.
The second row shows the optical flow field obtained by the FOVOF model. The edge discontinuity is reserved to some extent, but the contour of the object is not very clear.
In the third row, the optical flow field obtained by the DFOVOF optical flow model is presented. The contour edge is much more obvious; meanwhile the details of image texture can be seen clearly. The DFOVOF model expands the brightness constraint equation in the form of a Taylor fractionalorder, and the local brightness correlation of the image is also added for the calculation of the optical flow field, which further reduces the error and improves the accuracy of the model.
In order to evaluate the influence of fractionalorder parameters on the performance of the model and find out the correlation between the fractionalorder and the image characteristics, the AVAE/AVEE of the optical flow field of each image sequence at different fractional orders is estimated, as shown in Figure 2. At this time we choose . The smaller the value of AVAE/AVEE, the higher the accuracy of the optical flow field estimation. The first row is the AVAE and AVEE of image sequence “Yosemite”; the second row is the AVAE and AVEE of image sequence “Army.” The third row is the AVAE and AVEE of image sequence “Urban.”
We can find in Figure 2 that the minimum AVAE/AVEE is distributed in a range of and the minimum AVAE/AVEE is in a range of ; the simpler the image texture, the smoother the surface representing the variation of the value of AVEE/AVAE to variable and ; the more complex the image texture, the steeper the surface; the surface would also increase in incline when there exists a nonrigid motion in image sequences. Thus, it is clear that the algorithm presented here can improve the accuracy of motion estimation, especially for image sequences with complex texture and nonrigid motion. In conclusion, we choose a small for a complex texture image and choose a relatively large for an simple texture image sequence. What is more, nonrigid motion would also make the optimal small. The optimal changes little in different image sequence.
To test the effect of the mask window size on the algorithm’s performance and find out the correlation between mask window length and image features, we calculate the AVEE of optical flow estimation of six image sequences at different mask window size which was shown in Table 1, where , , where (a) is Venus; (b) is dimetrodon; (c) is hydrangea; (d) is Rubberwhale; (e) is grove; and (f) is urban. For comparison, the AVEE of HS model and FOVOF model were also added to the table. For comparison, the AVEE values for the optical flow field estimation of the HS model and the FOVOF model are also added to the table.

We can see in Table 1 that the value of AVEE reduces further when applied to the DFOVOF model; in the DFOVOF model the error rate becomes smaller when the mask window size increases—in general, we get the smallest AVEE value when is within (2, 7). For all the images, AVEE will decrease first when rises; after AVEE arrives at its optimal value, it will then increase when rises. The more complex the image texture is or the more nonrigid the motion is, the smaller the value of will be.
In conclusion, we should choose a large value for relatively simple image sequences and choose a small value for complex texture images and nonrigid motion images.
5. Conclusion
We have presented an improved fractionalorder optical flow model, which applies fractionalorder both in brightness constraint equation and in smoothness constraint equation of the HS model. What is more, we have proposed twodimensional fractional differential masks and used them to simplify the computation of dual fractionalorder derivatives. Experiments show that the model created here improves accuracy further. We have also found the relationships between the values of , , and and the image content. So different image content should use different , , and , but the algorithm we employ utilizes the same , , and in the whole image. Further studies will focus on the adaptive adjustments of the value of , , and in relation to the image content.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (61503141), Guangdong Province Frontier and Key Technological Innovation Special Funds (2016B090912001), the Production Research Project in Guangzhou (201604010114), the International Cooperation Projects of Science and Technology Information Bureau in Guangzhou City (2012J5100001), Sea Defense Public Welfare Project (201505002), and the Fundamental Research Funds for the Central Universities.
References
 B. K. P. Horn and B. G. Schunck, “Determining optical flow,” Artificial Intelligence, vol. 17, no. 1–3, pp. 185–203, 1981. View at: Publisher Site  Google Scholar
 L. Xu, J. Jia, and Y. Matsushita, “Motion detail preserving optical flow estimation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 9, pp. 1744–1757, 2012. View at: Publisher Site  Google Scholar
 Z. Chen, H. Jin, Z. Lin, S. Cohen, and Y. Wu, “Large displacement optical flow from nearest neighbor fields,” in Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2013, pp. 2443–2450, USA, June 2013. View at: Publisher Site  Google Scholar
 X. Zhang, H.M. Hu, F. Jiang, and B. Li, “Pedestrian detection based on hierarchical cooccurrence model for occlusion handling,” Neurocomputing, vol. 168, pp. 861–870, 2015. View at: Publisher Site  Google Scholar
 W. Li, D. Cosker, Z. Lv, and M. Brown, “Nonrigid optical flow ground truth for realworld scenes with timevarying shading effects,” IEEE Robotics and Automation Letters, vol. 2, no. 1, pp. 231–238, 2017. View at: Publisher Site  Google Scholar
 W. Li, D. Cosker, and M. Brown, “Drift robust nonrigid optical flow enhancement for long sequences,” Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology, vol. 31, no. 5, pp. 2583–2595, 2016. View at: Publisher Site  Google Scholar
 X.G. Lv, Y.Z. Song, S.X. Wang, and J. Le, “Image restoration with a highorder total variation minimization method,” Applied Mathematical Modelling, vol. 37, no. 1617, pp. 8210–8224, 2013. View at: Publisher Site  Google Scholar
 T. Chan, A. Marquina, and P. Mulet, “Highorder total variationbased image restoration,” Society for Industrial and Applied Mathematics, vol. 22, no. 2, pp. 503–516, 2000. View at: Publisher Site  Google Scholar  MathSciNet
 Q. Yang, D. Chen, T. Zhao, and Y. Chen, “Fractional calculus in image processing: a review,” Fractional Calculus and Applied Analysis, vol. 19, no. 5, pp. 1222–1249, 2016. View at: Publisher Site  Google Scholar  MathSciNet
 D. Chen, Y. Chen, and D. Xue, “Fractionalorder total variation image denoising based on proximity algorithm,” Applied Mathematics and Computation, vol. 257, pp. 537–545, 2015. View at: Publisher Site  Google Scholar  MathSciNet
 Y.F. Pu, J.L. Zhou, and X. Yuan, “Fractional differential mask: a fractional differentialbased approach for multiscale texture enhancement,” IEEE Transactions on Image Processing, vol. 19, no. 2, pp. 491–511, 2010. View at: Publisher Site  Google Scholar  MathSciNet
 D. Chen, H. Sheng, Y. Q. Chen, and D. Xue, “Fractionalorder variational optical flowmodel for motion estimation,” Philosophical Transactions of the Royal Society A: Mathematical, Physical & Engineering Sciences, vol. 371, no. 1990, Article ID 20120148, 2013. View at: Publisher Site  Google Scholar
 P. Kumar, S. Kumar, and B. Raman, “A fractional order variational model for the robust estimation of optical flow from image sequences,” Optik  International Journal for Light and Electron Optics, vol. 127, no. 20, pp. 8710–8727, 2016. View at: Publisher Site  Google Scholar
 P. Kumar, S. Kumar, and B. Raman, “A fractional order variational model for tracking the motion of objects in the applications of video surveillance,” in Proceedings of the 8th International Conference on Advanced Computational Intelligence, ICACI 2016, pp. 117–123, Thailand, February 2016. View at: Publisher Site  Google Scholar
 P. Kumar and S. Kumar, “A modified variational functional for estimating dense and discontinuity preserving optical flow in various spectrum,” AEÜ  International Journal of Electronics and Communications, 2015. View at: Publisher Site  Google Scholar
 S. G. Bardeji, I. N. Figueiredo, and E. l. Sousa, “Optical flow with fractional order regularization: variational model and solution method,” Applied Numerical Mathematics, vol. 114, pp. 188–200, 2017. View at: Publisher Site  Google Scholar  MathSciNet
 K. S. Miller, “Derivatives of noninteger order,” Mathematics Magazine, vol. 68, no. 3, pp. 183–192, 1995. View at: Publisher Site  Google Scholar  MathSciNet
 L. Podlubny, Fractional Differential Equations: An Introduction to Fractional Derivatives, Fractional Differential Equations, to Methods of Their Solution And Some of Their Applications, Academic Press, 1999. View at: MathSciNet
 O. P. Agrawal, “Formulation of eulerlagrange equations for fractional variational problems,” Journal of Mathematical Analysis and Applications, vol. 272, no. 1, pp. 368–379, 2002. View at: Publisher Site  Google Scholar  MathSciNet
 S. Baker, D. Scharstein, J. Lewis, and R. Szeliski, “A database and evaluation methodology for optical flow,” International Journal of Computer Vision, vol. 92, no. 1, pp. 1–31, March 2011. View at: Publisher Site  Google Scholar
Copyright
Copyright © 2018 Bin Zhu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.