An Object Detection Method Using Wavelet Optical Flow and Hybrid Linear-Nonlinear Classifier

Abstract

We propose a new computational intelligence method using wavelet optical flow and hybrid linear-nonlinear classifier for object detection. With the existing optical flow methods, it is difficult to accurately estimate moving objects with diverse speeds. We propose a wavelet-based optical flow method, which uses wavelet decomposition in optical flow motion estimation. The algorithm can accurately detect moving objects with variable speeds in a scene. In addition, we use the hybrid linear-nonlinear classifier (HLNLC) to classify moving objects and static background. HLNLC transforms a nonoptimal scalar variable into its likelihood ratio and uses a scalar quantity as the decision variable. This approach is appropriate for the classification of optical flow feature vectors with unequal variance matrices. The experimental results confirm that our proposed object detection method has an improved accuracy and computation efficiency over other state-of-the-art methods.

1. Introduction

In modern engineering, the requests for research and design are increasingly achieved with the help of intelligent models. Computational intelligence (CI) has emerged as powerful tools for information processing, decision making, and knowledge management [1]. CI is a set of nature-inspired computational methodologies and approaches to address complex real-world problems to which traditional approaches [2, 3]. In this paper, we propose a new computational intelligence method using wavelet optical flow and hybrid linear-nonlinear classifier (HLNLC) for object detection. Object detection can be subcategorized as either the detection that has similar characteristics [4, 5] or the detection of a specific object in a video sequence [6, 7]. Our paper is focused on the second category.

One important task in object detection is motion estimation. Optical flow is one commonly used approach to estimate the object motion. Starting with the original algorithms by Lucas and Kanade (LK) [8] as well as Horn and Schunck (HS) [9], gradient-based methods have led to other improved optical flow estimation methods. However, when the image background is cluttered or the detected object is moving at high speed, the accuracy of gradient-based methods will be significantly decreased [10]. Another important task in object detection is classification. Classifier techniques such as pulse-coupled neural network (PCNN) [11], fuzzy neural network (FNN) [12], Gaussian SVM (GSVM) [13], and linear discriminant analysis (LDA) [14] have been applied to different object detection situations. In PCNN and FNN schemes, detection accuracy may be coupled with small training errors because each image pixel is associated with a unique neuron and vice versa. In SVM classifier approach, the classifier is characterized by excessive complexity when it comes to the binary classification task of identifying moving objects versus static background. The LDA classifier is a more robust linear classifier which has proven itself as the ideal observer for input feature vectors with equal covariance matrices [15], but it is not the optimal choice for data that contain multivariate optical flow vectors with unequal covariance matrices.

Our work will consider both factors. We propose a new object detection method using wavelet-based optical flow and hybrid linear-nonlinear classifier. Some wavelet-based optical flow estimation approaches have been proposed; Wu et al. [16] used wavelets to model and reconstruct flow vectors; at each iteration, the estimation will be repeated, which is to reduce efficiency. In [17], Bernard assumed that the optical flow was locally constant. In [18], Srinivasan and Chellappa proposed a similar method which modeled optical field using a set of overlapping basis functions. In [17, 18], optical flow computation has been simplified, and the disadvantage is to compromise the accuracy, especially when several objects with different speeds existed in one scene. In this paper, we use wavelet calculus to compute derivatives of the functions in terms of the scaling expansion coefficients. Our proposed method could achieve an accelerated optical flow computation and accurately estimate the motions of different speed moving objects in the same scenes.

In the binary classification about distinguishing moving objects versus static background, a linear classifier is a more robust choice than existing methods [19]. Linear discriminant analysis (LDA) is the most popular linear classifier. LDA produces accurate classifications for two types of input feature matrices: those with normal distributions and those with equal covariance. However, it has trouble with vector data that have an unequal covariance. Thus, we use the novel hybrid linear-nonlinear classifier (HLNLC). HLNLC was proposed by Chen et al. in 2010 [20]. It has been proved to be a more robust linear classifier than LDA and other existing classifiers.

This paper is organized as follows. Section 2 describes wavelet-based optical flow motion estimation method. Section 3 introduces the hybrid linear-nonlinear classifier. Section 4 describes rectangle window scan algorithm. Section 5 describes experimental results. Our conclusions are presented in Section 6.

2. Wavelet Based Optical Flow

2.1. Gradient-Based Optical Flow

Gradient-based optical flow algorithms are based on the assumption of constant brightness [8, 9], where it is assumed that the gradient value of a pixel will not vary due to displacement [21]. It can be described as 𝐸=𝐼_𝑥𝑢_𝑥+𝐼_𝑦𝑣_𝑦+𝐼_𝑡²+𝑎|Δ𝑢|²+|Δ𝑣|²𝑑𝑥𝑑𝑦. (1)

Here, 𝐼(𝑥,𝑦,𝑡) represents the brightness value of pixel (𝑥,𝑦) at time 𝑡. 𝐼_𝑥, 𝐼_𝑦, and 𝐼_𝑡 are the partial derivatives of 𝐼(𝑥,𝑦,𝑡) with respect to 𝑥, 𝑦, and 𝑡. The variable (𝑣_𝑥,𝑣_𝑦) is the velocity vector of optical flow estimation about 𝐼(𝑥,𝑦,𝑡), and (𝑢,𝑣) is the gradient operator of 𝐼(𝑥,𝑦,𝑡). The brightness constancy assumption assumes that the motion vectors are constant within small windows and that the image sequence 𝐼(𝑥,𝑦,𝑡) will not change significantly during a short period of time. This assumption can be expressed as Δ𝐼𝑥,𝑦,𝑡=Δ𝐼𝑥+𝑢,𝑦+𝑣,𝑡,|Δ𝑢|²+|Δ𝑣|²=0. (2)

Based on (1) and (2), the flow algorithm produces two simultaneous equations for the velocity vector 𝑣_𝑥 and 𝑣_𝑦: 𝑝_𝑥𝑣_𝑥+𝑝_𝑥𝑦𝑣_𝑦+𝑝_𝑥𝑡𝑝=0,_𝑥𝑦𝑣_𝑥+𝑝_𝑦𝑣_𝑦+𝑝_𝑦𝑡=0, (3)𝑝_𝑥, 𝑝_𝑥𝑦, 𝑝_𝑦, 𝑝_𝑥𝑡, and 𝑝_𝑦𝑡 are the product of the partial derivatives 𝐼_𝑥, 𝐼_𝑦 in a time range 𝐼_𝑡. The detailed representations may be written as 𝑝_𝑥=_𝑡𝐼_𝑥𝐼_𝑥,𝑝_𝑥𝑦=_𝑡𝐼_𝑥𝐼_𝑦,𝑝_𝑦=_𝑡𝐼_𝑦𝐼_𝑦,𝑝_𝑥𝑡=_𝑡𝐼_𝑥𝐼_𝑡,𝑝_𝑦𝑡=_𝑡𝐼_𝑦𝐼_𝑡. (4)

In gradient-based optical flow algorithms, object displacement between successive frames determines the accuracy of optical flow estimation because the assumption that there will be no major changes between successive frames will break down when the displacement between frames is significant. In order to improve the accuracy of optical flow estimation, displacements between video frames should be projected and recalculated in different frame rates, which means that the algorithm should be able to adaptively adjust the velocity components for different objects moving with different speeds.

2.2. Wavelet Based Optical Flow Estimation

We try to apply the wavelet transform into optical flow estimation. Wavelet transform is an important tool for signal processing, and its magnitude will not oscillate around singularities as the transform magnitude is locally nearly shift invariant [22]. To apply the wavelet decomposition to optical flow estimation, we transform the optical flow equation into the following expression [23]: 𝐼=𝐼²_𝑥𝑢²+𝐼²_𝑦𝑣²+𝐼²_𝑡+2𝐼_𝑥𝐼_𝑦𝑢𝑣+2𝐼_𝑥𝐼_𝑡𝑢+2𝐼_𝑦𝐼_𝑡𝑣+𝜆𝑢²_𝑥+𝑢²_𝑦+𝑣²_𝑥+𝑣²_𝑦𝑑𝑥𝑑𝑦. (5)

Suppose the image size is 𝑀×𝑀 pixels. Then the optical flow vector [𝑢(𝑥,𝑦),𝑣(𝑥,𝑦)] can be expressed as 𝑢𝑥,𝑦=^𝑀−1_𝑚=0^𝑀−1_𝑛=0,𝑣=𝑢⋅𝛿𝑥−𝑚,𝑦−𝑛𝑥,𝑦^𝑀−1_𝑚=0^𝑀−1_𝑛=0𝛿()=𝑝(𝑋)=()𝑣⋅𝛿𝑥−𝑚,𝑦−𝑛𝑋,𝑌𝑚,𝑛∣𝑌𝐷^𝑌_{𝑚,𝑛𝑑∈𝐷𝐿(𝑦)𝑚,𝑛}ℓ_𝑑𝑠_𝑢𝑦−𝑥𝑠. (6)

The variables 𝑢(𝑥,𝑦), 𝑣(𝑥,𝑦) are the weighted coefficients of the optical flow. Once 𝑢_𝑚,𝑛 and 𝑣_𝑚,𝑛 can be determined, the optical flow estimation will be accomplished [24]. Thus, we transform the optical flow estimation into a calculation that includes 2𝑀² node variables, where 𝑢_𝑚,𝑛 and 𝑣_𝑚,𝑛 can minimize the object function (5).

𝐼²_𝑥, 𝐼²_𝑦, and 𝐼_𝑥𝐼_𝑦 denote the products of spatial derivatives: 𝐼²_𝑥𝑥,𝑦=^𝑀−1_𝑚=0^𝑀−1_𝑛=0𝑎_𝑚,𝑛,𝐼⋅𝛿𝑥−𝑚,𝑦−𝑛_2𝑦𝑥,𝑦=^𝑀−1_𝑚=0^𝑀−1_𝑛=0𝑏_𝑚,𝑛,𝐼⋅𝛿𝑥−𝑚,𝑦−𝑛_𝑥𝐼_𝑦𝑥,𝑦=^𝑀−1_𝑚=0^𝑀−1_𝑛=0𝑐_𝑚,𝑛⋅𝛿𝑥−𝑚,𝑦−𝑛. (7)(8)(9)𝐼²_𝑡, 𝐼_𝑥𝐼_𝑡, and 𝐼_𝑦𝐼_𝑡 are products of spatial and temporal partial derivatives, where the time variable 𝑡 is taken from the variable frame rate: 𝐼²_𝑡𝑥,𝑦=^𝑀−1_𝑚=0^𝑀−1_𝑛=0𝑑_𝑚,𝑛,𝐼⋅𝛿𝑥−𝑚,𝑦−𝑛_𝑥𝐼_𝑡𝑥,𝑦=^𝑀−1_𝑚=0^𝑀−1_𝑛=0𝑒_𝑚,𝑛,𝐼⋅𝛿𝑥−𝑚,𝑦−𝑛_𝑦𝐼_𝑡𝑥,𝑦=^𝑀−1_𝑚=0^𝑀−1_𝑛=0𝑓_𝑚,𝑛⋅𝛿𝑥−𝑚,𝑦−𝑛. (10)

The shortest frame interval is 𝜎, and the interval of multiplications is set to 𝑛𝜎, so that the product of 𝑝_𝑥(𝑡) and 𝑝_𝑦(𝑡) in an interval 𝑛𝜎 is given by 𝑝_𝑥(𝑡)=_𝑚,𝑛𝐼_𝑥𝑢_𝑚,𝑛𝑣_𝑚,𝑛𝑝⋅𝜑𝑥,𝑦∣𝑡+𝑛𝜎𝑑𝑥𝑑𝑦,_𝑦(𝑡)=_𝑚,𝑛𝐼_𝑦𝑢_𝑚,𝑛𝑣_𝑚,𝑛𝜑⋅𝜑𝑥,𝑦∣𝑡+𝑛𝜎𝑑𝑥𝑑𝑦,𝑥,𝑦∣𝑡=𝑤_𝐿⋅𝐾𝐿−𝐿^′, (11) where 𝑤_𝐿=∑^𝑇_𝑡=1∑_{𝑑∈𝐷𝑌𝑚,𝑛}(𝑝(𝐿_𝑡(𝑦))/𝑇|𝐷^𝑌_𝑚,𝑛|)ℓ_𝑑(𝑠_𝑢(𝑦−𝑥)/𝑠).

The partial derivatives of 𝐼(𝑥,𝑦,𝑡) can be obtained via two difference equations: 𝐼_𝑥,𝐼=𝐼𝑥+1,𝑦,𝑡−𝐼𝑥,𝑦,𝑡_𝑦=𝐼𝑥,𝑦+1,𝑡−𝐼𝑥,𝑦,𝑡. (12)

In the time interval 𝑛𝜎, we compute the optical flow partial derivatives using (7). In our algorithm, 𝑝_𝑥(𝑡) and 𝑝_𝑦(𝑡) are replaced by the approximate products ~𝑝_𝑥(𝑡) and ~𝑝_𝑦(𝑡) in order to improve computation accuracy. The specific formulas are shown in the following expressions: 𝑝_𝑥(𝑡)≈^𝑛−1_𝑖=01𝑛_𝑡𝐼_𝑥𝑢_𝑚,𝑛𝑣_𝑚,𝑛⋅𝜑^′𝑥,𝑦∣𝑡+𝑖𝑑𝑥𝑑𝑦≡∼𝑝_𝑥(𝑡),𝑝_𝑦(𝑡)≈^𝑛−1_𝑖=01𝑛_𝑡𝐼_𝑦𝑢_𝑚,𝑛𝑣_𝑚,𝑛⋅𝜑^′𝑥,𝑦∣𝑡+𝑖𝑑𝑥𝑑𝑦≡∼𝑝_𝑦(𝑡),𝜑^′=1𝑥,𝑦∣𝑡𝐷^𝑌_𝑚,𝑛ℓ_𝑑𝑠_𝑢𝑦−𝑥𝑠⋅𝑝𝑐𝑦=𝑝_𝑚,𝑛∣𝐿𝑦. (13)

In this work, continuous displacements between image frames are considered to be very small. Specifically, 𝐼_𝑥 and 𝐼_𝑦 will be considered as constants in the time interval 𝑡~𝑡+𝑛𝜎. With this assumption, we can replace 𝑝_𝑥(𝑡) and 𝑝_𝑦(𝑡) with ~𝑝_𝑥(𝑡) and ~𝑝_𝑦(𝑡). The amplitude of optical flow is estimated by considering the previous frame 𝑡−𝜎. The adaptive parameter 𝑛 is determined by the optical flow estimation (𝑉_𝑥(𝑡−𝜎),𝑉_𝑦(𝑡−𝜎)) from the previous frame. It can be used to estimate the pseudovariable frame rate (𝑣_𝑥,𝑣_𝑦). When computing the product in frame interval 𝑛𝜎, 𝑛 is adjusted automatically according to the speed of detected object. For objects moving at high speed, 𝑛 is small, while, for slow-moving objects, it is larger. Furthermore, 𝑛(𝑣_𝑥,𝑣_𝑦) may take on different values at any pixel location because (𝑣_𝑥,𝑣_𝑦) may exhibit spatial variability in the optical flow estimation algorithm. By using a wavelet transform, we can rewrite (5): 𝐸=𝑎_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦⋅_𝑚,𝑛𝑢_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦²+𝑏_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦⋅_𝑚,𝑛𝑣_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦²+2𝑐_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦⋅_𝑚,𝑛𝑢_𝑚,𝑛𝛿_𝑚,𝑛⋅𝑥,𝑦_𝑚,𝑛𝑣_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦+𝑑_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦+2_𝑚,𝑛𝑒_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦⋅_𝑚,𝑛𝑢_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦+2_𝑚,𝑛𝑓_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦⋅_𝑚,𝑛𝑣_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦+𝜆_𝑚,𝑛𝑢_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦²+_𝑚,𝑛𝑣_𝑚,𝑛𝛿_𝑚,𝑛𝑥,𝑦²𝑑𝑥𝑑𝑦. (14)

Figure 1 depicts the application of wavelet into optical flow estimation.

Figure 1 

Application of wavelet calculations into optical flow estimation.

The optimal coefficient of (𝑢_𝑚,𝑛,𝑣_𝑚,𝑛) can be determined by the following expression: ()=((−𝑢Coeff𝑚,𝑛1−𝐸𝑚,𝑛))exp𝑥,𝑦−𝑣𝑥,𝑦²𝜎², (15) where 𝑚, 𝑛=0,1,…,𝑀−1.

The sparse representation is 𝐸𝑢=𝑏, and the matrix obtained by the wavelet transform is 𝐸∈𝑅^𝑀2×𝑀2. Elements of matrix 𝐸 are written as 𝐸()𝑢𝑚,𝑛=arccos^𝑚𝑢_𝑛+𝑣_𝑚𝑣_𝑛+1𝑢²_𝑚+𝑣²_𝑚+1𝑢²_𝑛+𝑣²_𝑛+1. (16)

The combination of wavelet transform and optical flow defines moving objects via a sparse linear representation in the defined structure. The wavelet algorithm has collected all the information from the optical flow estimation and stored it in the matrix 𝐸. Once the computation 𝐸𝑢=𝑏 is complete, an optimal and sparse (𝑢_𝑚,𝑛,𝑣_𝑚,𝑛) will have been determined. Our method transforms the optical flow estimation into the problem of minimizing an energy function. The coefficients determination yields accurate optical flow vectors.

3. Classification

3.1. Hybrid Linear-Nonlinear Classifier

The novel hybrid linear-nonlinear classifier (HLNLC) divides the traditional binary classification into two stages: in the first stage, a linear function combines the input feature vector and a scalar variable, and in the second stage, the scalar variable is transformed into a decision variable [25].

In a two-class sorting approach, a particular data set is divided into positive and negative parts. Suppose the feature vector is given by 𝑥=(𝑥₁,𝑥₂,…,𝑥_𝑁)^𝑇, where 𝑥 represents the joint outcome of some random variables, where there are 𝑁 of such variables. The corresponding distributions are described by a multivariable positive normal distribution (where the mean is 𝑢_𝑝=(𝑢_𝑝1,𝑢_𝑝2,…,𝑢_𝑝𝑁)^𝑇 and the 𝑁×𝑁 covariance matrix is [𝐶_𝑝]) and by a multivariable negative normal distribution (where the mean is 𝑢_𝑛=(𝑢_𝑛1,𝑢_𝑛2,…,𝑢_𝑛𝑁)^𝑇 and the 𝑁×𝑁 covariance matrix is [𝐶_𝑛]). The probability density function (PDF) for positive and negative components is (𝑥∣𝑝)∼𝑁(𝑢_𝑝𝑖;[𝐶_𝑝]) and (𝑥∣𝑛)∼𝑁(𝑢_𝑛𝑖;[𝐶_𝑛]).

In first stage of the HLNLC, the feature vector 𝑥 is mapped into a scalar vector 𝑦. Because 𝑥 follows the multivariable normal distribution and 𝑦 is a linear combination of 𝑥, we see that 𝑦 follows a two-variable normal distribution: 𝑦𝑝∼𝑁𝑢_𝑝;𝜎²_𝑝,𝑦𝑛∼𝑁𝑢_𝑛;𝜎²_𝑛. (17)

These parameters can be represented by the input vector 𝑣 and the other related input parameters. They may be written as 𝑢_𝑝=𝑣^𝑇𝑢_𝑝;𝜎²_𝑝=𝑣^𝑇𝐶_𝑝𝑢𝑣,_𝑛=𝑣^𝑇𝑢_𝑛;𝜎²_𝑛=𝑣^𝑇𝐶_𝑛𝑣. (18)

In [26], the classifier is improved by projecting the multivariable classification algorithm into a two-density distribution interval, with the projection method described by 𝐷(𝑥)=12𝑥−𝑢_𝑝^𝑇𝐶_𝑝⁻¹𝑥−𝑢_𝑝−12𝑥−𝑢_𝑛^𝑇𝐶_𝑛⁻¹𝑥−𝑢_𝑛+12𝐶ln^𝑝𝐶_𝑛. (19)

In a second stage of the HLNLC algorithm, the likelihood ratio of 𝑦 is used as a decision variable: 𝐿𝑅_𝑦=𝑓𝑦∣𝑝𝑓=𝜎𝑦∣𝑛𝜎^𝑝_𝑛−1exp2𝑏𝑦−𝑎²−𝑦². (20)

Based on binormal ROC theory, the corresponding 𝑃_AUC can be expressed as 𝑃_AUC𝑑=Φ^𝑎√2−𝑑+2𝐹^𝑎√2,0;−1−𝑐²1+𝑐², (21) where the 𝑑_𝑎 and 𝑐 are 𝑑_𝑎=√2𝑢_𝑝−𝑢_𝑛𝜎²_𝑝+𝜎²_𝑛𝜎,𝑐=^𝑝−𝜎_𝑛𝜎_𝑝+𝜎_𝑛. (22)

𝐹(𝑋,𝑌;𝜌) is the cumulative distribution function (CDF) of the standard normal distribution. Using the parameter (𝑢_𝑝,𝐶_𝑝,𝑢_𝑛,𝐶_𝑛), we may express the AUC as a function of the linear coefficient vector 𝑣: AUC_HLNLC𝑑=Φ^𝑎(𝑣)√2−𝑑+2𝐹^𝑎(𝑣)√2(𝑣),0;−1−𝑐²1+𝑐(𝑣)²,𝑑_𝑎(𝑣)=√2𝑣^𝑇𝑢_𝑝−𝑢_𝑛𝑣^𝑇𝐶_𝑝+𝐶_𝑛𝑣,𝑐(𝑣)=𝑣^𝑇𝐶_𝑝𝑣−𝑣^𝑇𝐶_𝑛𝑣𝑣^𝑇𝐶_𝑝𝑣+𝑣^𝑇𝐶_𝑛𝑣. (23)(24)(25)

In order to find the optimal linear function of the HLNLC, which mainly focuses on 𝑣 and AUC_HLNLC, the HLNLC algorithm classifies the optimization problem as follows: 𝑣_HLNLC=argmax_𝑣AUC_HLNLC(𝑣). (26)

The optimization problem is given by 𝜕AUC_HLNLC/𝜕𝑣=0, which can be solved with gradient-based mathematical methods. The specific process is shown in the following expressions: 𝜕AUC_HLNLC𝜕𝑣_𝑖=𝜕AUC_HLNLC𝜕𝑑_𝑎𝜕𝑑_𝑎𝜕𝑣_𝑖+𝜕AUC_HLNLC𝜕𝑐𝜕𝑐𝜕𝑣_𝑖,𝜕AUC_HLC𝜕𝑑_𝑎=exp−𝑑²_𝑎/42√𝜋,(𝑐≠0),exp−𝑑²_𝑎/42√𝜋1−2Φ𝜌𝑑_𝑎21−𝜌²,(𝑐≠0),𝜕𝑑_𝑎𝜕𝑣_𝑖=2(𝑣⋅Δ𝑢)𝑣^𝑇𝐶_𝑝+𝐶_𝑛𝑣−2(𝑣⋅Δ𝑢)∑^𝑛_𝑖=1𝐶_𝑖𝑗𝑣_𝑗𝑑_𝑎𝑣^𝑇([𝐶])𝑣²,𝜕AUC_HLNLC=(),(𝑐)𝜕𝑐0,𝑐≠02sign𝜋1+𝑐²−𝑑exp^2𝑎41−𝜌²,(𝑐≠0),𝜕𝑐𝜕𝑣_𝑖=2𝜎_𝑝+𝜎_𝑛²𝜎_𝑝∑^𝑛_𝑖=1𝐶^𝑛_𝑖𝑗𝑣_𝑖−𝜎_𝑛∑^𝑛_𝑖=1𝐶^𝑝_𝑖𝑗𝑣_𝑖𝜎_𝑝−𝜎_𝑛. (27)

3.2. Classification of Optical Flow Vectors

In this paper, the linear coefficient vector 𝑣 is determined by the parameter 𝜃 for a 2D optical flow vector (𝑢_𝑚,𝑛,𝑣_𝑚,𝑛), which can be understood to be the angle between vector coordinate 𝑥 and vector coordinate 𝑦. Any 2D optical flow vector can be normalized as 𝑦=(𝑣)^𝑇𝑥=𝑢_𝑚,𝑛cos𝜃+𝑣_𝑚,𝑛sin𝜃. (28)

The operating assumption in the HLNLC algorithm is that two types of feature data follow a pair of multivariable normal distributions. However, the optical flow distributions contain some small differences. We can implement a more robust method. The pair of optical flow vector variables uses bivariate normal distributions to normalize the data. Specifically, 𝑢_𝑚,𝑛,𝑣_𝑚,𝑛1∣𝜌∼𝑁𝑢²_𝑚,𝑛𝜌_𝑢,𝑣(𝑢⋅𝑣)_𝑚,𝑛𝜌_𝑢,𝑣(𝑢⋅𝑣)_𝑚,𝑛1𝑣²_𝑚,𝑛. (29)

Here, 𝜌_𝑢,𝑣 represents feature covariance of the optical flow (𝑢_𝑚,𝑛,𝑣_𝑚,𝑛). Using this method, (𝑢_𝑚,𝑛,𝑣_𝑚,𝑛) is normalized into the normal distribution. The parameters (𝑢_𝑝,𝜎_𝑝,𝑢_𝑛,𝜎_𝑛) that concern the HLNLC algorithm can be obtained with (18), in which positive values indicate motion areas and negative values represent background. AUC_HLNLC can be calculated by using (23). The related parameter (𝑢(𝜃_HLNLC),𝑣(𝜃_HLNLC)) is used to produce an ROC curve and calculate AUC_HLNLC.

4. Rectangle Window Scan

In order to detect moving objects, an 𝑀×𝑁 rectangle window should be determined. We propose rectangle window scan algorithm; in scanning process, the classified optical flow vectors which are attributed to moving object are the input variables, and a rectangle window will shift 𝑛 pixel locations in each direction per unit time. The operation is repeated until the rectangle window size is less than the presupposed threshold value. The detection area marked by the rectangle window will be the final output. Our proposed method could detect multiple moving objects in one scene. However, the iterative process may be time consuming. Figure 2 illustrates the details of rectangle window scan algorithm.

Figure 2 

Rectangle window scan method.

In the method, the 𝑖th focused area at the 𝑗th scan line is denoted by 𝐹(𝑖,𝑗), and the related rectangle window is recorded as RW_{𝐹(𝑖,𝑗)}. Initially, the scan area RW_𝐹(1,1) is obtained by a normal adaptive modification calculation method. Thereafter, the rectangle window shifts to the right by 𝑥 pixels, where 𝑥=10 was used in this paper, and RW_𝐹(1,1) is shifted to RW_𝐹(1,2). There is no need to recalculate the overlap region of an integral scan because the two adjacent regions RW_𝐹(1,1) and RW_𝐹(1,2) share the same overlap region (RW_1−𝑥)×𝑁. For the remaining horizontal scan lines, 𝐹(𝑖,𝑥) describes the region of concern which is below 𝐹(𝑖−1,𝑥). The upper and lower boundaries of the nonoverlapping region are defined as 𝐻(𝑖,𝑥) and 𝐿(𝑖,𝑥).

Based on the distribution principle for motion vectors, less than 50% of the motion vectors may be zero vectors. We use the sum of absolute gradient difference (SAGD) to judge motion vector value. SAGD can be calculated in the following expression: SAGD(𝑢,𝑣)=max^{𝑀∑𝑁𝑥=0∑𝑦=0𝑢𝑖+𝑥,𝑗+𝑦−𝑢𝑖+𝑥+𝑎,𝑗+𝑦+𝑎}_{𝑀∑𝑁𝑥=0∑𝑦=0𝑣𝑖+𝑥,𝑗+𝑦−𝑣𝑖+𝑥+𝑎,𝑗+𝑦+𝑎}. (30)

Depending on the current location of rectangle window, some adjacent windows may not exist. We use windows adjacent to block combination to estimate the object motion, using the formula 𝐻_𝐹(𝑖,1)=𝐻_{𝐹(𝑖−1,1)}−SAG_{𝐻(𝑖−1,1)}+SAG_{𝐿(𝑖−1,1)}. (31)

For the remaining areas, 𝐹(𝑖,𝑗), with 𝑖, 𝑗=2,3,…,𝐻_{𝐹(𝑖,𝑗)}, the moving object region is identified as RW=^𝑀𝑁_𝑖=1𝑗=1𝐻_{𝐹(𝑖,𝑗−1)}−𝐻_{𝐹(𝑖,𝑗)}+𝐿_{𝐹(𝑖,𝑗)}. (32)

5. Experimental Results

In this section, we validate the performance of our proposed algorithm on four different videos. All of the source videos were presented in a consistent format (MPEG-2 standard, 25.68 frames per second (fps)). Video (a) was produced with cameras where high-speed moving cars appear in both close and distant scenes, it has 996 frames of 768 × 576 resolution. Video (b) is a standard video compression sequence known as the coastguard sequence, where a video camera is fixed on a moving boat so it appears that the background is moving; it has 876 frames of 768 × 576 resolution. Video (c) is a spatial satellite video sequence, and the satellite is the detected object; it has 1025 frames of 720 × 480 resolution. Video (d) is a human-motion video in which the motion is a human running; it has 825 frames of 720 × 480 resolution. These four videos are the most tested videos, in which the pedestrians, vehicles, and satellites are typical detected targets. Comparison experiments with other state-of-the-art object detection algorithms could demonstrate the efficiency of our proposed method. Specific steps are shown in Figure 3.

Figure 3 

The whole experiment flowchart.

5.1. Optical Flow Experiments

We use a constant value 𝑎 to determine the time interval between video frames, and the parameter value 𝑎=0.64 pixel/ms was found to be suitable. Each frame was processed with subsampling and rescaled under the wavelet estimation. Two important steps are involved: (1) normalization of optical flow vectors and computation of the matrix 𝐸 and (2) computation of the spare representation of 𝐸𝑢=𝑏. Our results show that cars and human bodies can be detected in (a) and (d). Different frequency components in unstable regions and jitters have been removed in sequences (c). Compared with other video sequences, detected objects in sequence (b) have different characteristics. The optical flow is detected for background area. The experimental results are shown in Figure 4.

(a) Cars sequence

(b) Coastguard sequence

(c) Satellite sequence

(d) Human running sequence

Figure 4 

Optical flow estimation for different sequences.

Computation time comparisons results are shown in Table 1. We compare our proposed method with LK, HS, and occlusion-aware optical flow (OAOF) [27]. LK and HS calculations take almost the same time because they involve similar computation methods. OAOF takes a little longer time. Our proposed wavelet optical flow method has an improved computation time, with a reduction of nearly 5~6 sec over LK and HS and 10~14 sec over OAOF. It clearly demonstrates the efficiency of using wavelet transform in optical flow estimation.

Table 1 

Optical flow computation time comparison.

5.2. Classification Experiments

In the first stage of HLNLC algorithm, structure optimal parameters are chosen as 𝑢=0.64, and the classifier undergoes 20 iterations. In order to get a dense optical flow field, our method uses a global smoothness constraint. Each optical flow vector has a spatial connection with respect to high-speed moving object. Figure 5 shows the initial classification results. As background information and moving object features have similar characteristics, motion regions adhere and empty holes emerged. Obviously, classification result is not very satisfactory.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

Figure 5 

Initial classification results: (a), (b), (c), and (d) single frame in test video sequences; (e), (f), (g), and (h) corresponding classification result.

In the second stage, the classifier uses a decision variable to deal with topology changes in detection region. Figure 6 shows the classification results about video sequences (a) and (b). Motion areas in video (a) contain queues of cars, which belong to close scene and distant scene, respectively. Because the cars in distant scene are too wide, some background information has also been classified as motion region. Optical flow vectors generated from video (b) reflect background information. We employ a trust-region Newton-Rapthson method to solve this problem [28]. This method separates moving objects from background by making judgments about the most meaningful trust-region in sequences based on the maximum likelihood ratio.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

Figure 6 

Second stage classification results: (a), (b), (c), and (d) frames 7, 9, 11, and 25 of cars sequences (e), (f), (g), and (h) frames 95, 97, 98, and 100 of coastguard sequence.

We are interested in comparing the classification performance of HLNLC and other classifiers, including PCNN, FNN, GSVM, and LDA. We specified the population parameters of a pair of normal distributions (𝑢_𝑝,𝐶_𝑝,𝑢_𝑛,𝐶_𝑛) and drew samples from the specified distributions. Then, different classifiers are applied to each optical flow feature vector in the sampled data. The comparison results are shown in Table 2. M&SD are the mean and standard deviation of the motion region (positive region), and AUC is calculated based on M&SD: 𝑢AUC=Φ^𝑝−𝑢_𝑛𝜎²_𝑝+𝜎²_𝑛,DR=^𝑁_𝑖=1𝑏_𝑖√−2𝜋exp^𝑁_𝑖=1AUC_𝑖−𝜎²𝑁. (33)

Table 2 

Classification performance of PCNN, FNN, GSVM, LDA, and HLNLC.

The function Φ is the cumulative distribution function of the standard normal distribution. Positive values indicate motion areas, and negative values represent background. AUC, as a decision variable of normal distribution, equals the probability that outcomes from the actual positive class. A greater AUC means higher classification accuracy. Based on AUC, we were able to calculate the detection rate (DR), which is defined as the ratio between total moving object region and background area.

Our experiments use the mean AUC for each sequence which has different frame numbers, respectively. Neural network algorithms (PCNN⧵FNN) give rise to proliferation errors in the edges of moving objects and have larger deviations in their classification results. GSVM can directly classify data without using PCA, so the result is better than what is produced by NN. About LDA and HLNLC, we calculate AUC by inserting trained LDA vectors, and AUC_HLNLC is found to be superior to AUC_LDA. In all the methods, We observe that the HLNLC can substantially improve classification performance over other classifiers. M&SD, AUC, and DR curves obtained by different methods are shown in Figure 7.

(a)

(b)

(c)

Figure 7 

M&SD, AUC, and DR curves obtained by different methods.

5.3. Rectangle Window Scan Experiments

In this part, we used three other object detection methods for comparison. These included SIFT, background subtraction (BS), and Hough forest method (HF). The SIFT method is based on SIFT features, which is invariant to image scale and rotation [29, 30]. The background subtraction method uses temporal differencing pixels from a Laplacian model and completes the object detection task via a threshold value [31]. Hough forests can be regarded as a task-adapted codebook, in which different locations, scales, and motions are stored [32]. Figure 8 shows the experimental results; the object detection results obtained by our proposed method are shown with red solid boxes; the results in SIFT, BS, and HF are shown in blue, green, and pink dotted boxes. Our proposed method could detect moving objects in different distant scenes in the same sequence; in video sequence (a), moving cars in the distant scene were detected in green and orange solid rectangle windows. In addition, our proposed method could adaptively adjust the rectangle window that will be suitable for the detected object, such as satellite and human body in video sequences (c) and (d).

Figure 8 

Object detection results: (a)–(d) frames 7, 10, 14, and 17 of cars sequence, (e)–(h) frames 17, 26, 35, and 45 of coastguard sequence, (i)–(l) frames 27, 35, 54, and 69 of human running sequence, and (m)–(p) frames 29, 33, 40, and 44 of human walking sequence.

We use three measures to compare object detection accuracy [33]: OverLapRatio=^𝑁frame_𝑖=1𝐺^𝑡_𝑖∩𝑂^𝑡_𝑖𝐺^𝑡_𝑖∪𝑂^𝑡_𝑖,FDA(𝑡)=OverLapRatio𝑁^𝑡_𝐺+𝑁^𝑡_𝑂,/2SFDA=^𝑁frame_𝑖=1FDA(𝑡)∃𝑁^𝑡_𝐺OR𝑁^𝑡_𝑂,𝑚ODA=1−^𝑡+𝑓_𝑝𝑡𝑁^𝑡_𝐺,ODP=OverLapRatio𝑁^𝑡_mapped. (34)

Here, 𝐺^𝑡_𝑖 and 𝑂^𝑡_𝑖 represent 𝑖th ground-truth and detected object in frame 𝑡. 𝑁^𝑡_𝐺 and 𝑁^𝑡_𝑂 represent the number of “ground-truth” and detected object in frame 𝑡. 𝑓_𝑝𝑡 is the number of classification errors. 𝑚_𝑡 is the missed detection count. 𝑁^𝑡_mapped is the pair number of ground-truth and detected object in frame 𝑡. OverLapRatio is the quality of alignment between the detected objects and the ground-truth. SFDA is the detection accuracy for a video sequence, which is essentially the average of FDA over all of the relevant frames in the sequence; ODA is the object detection accuracy, which utilized the missed detection and classification error counts. ODP is the object detection precision, which gave us the precision of detection by taking into account the spatial overlap information between ground-truth and system output.

Experimental results are shown in Table 3. Compared with SIFT, our proposed algorithm increases SFDA, ODP, and ODA by about 3%~7%, 5%~10%, and >10%. Compared with BS and HF, the SFDA, ODP, and ODA are increased by about 5%~15%, 8%~15%, and 10%, respectively. It implies that our proposed algorithm has a higher detection accuracy and lower false detection rate. SFDA, ODP, and ODA curves obtained by different methods are shown in Figure 9.

Table 3 

Detection results of different methods.

(a)

(b)

(c)

Figure 9 

SFDA, ODA, and ODP curves obtained by different methods.

6. Conclusions

In this paper, we propose a new computational intelligence method for object detection. We apply the wavelet transform into optical flow estimation. Our proposed method is able to estimate the motions of different speed moving objects in the same scenes and achieve an accelerated optical flow computation. In the classification stage, HLNLC method is used to classify optical flow vectors, and it yields higher accuracy than other existing classifiers. Experimental results demonstrate that the combination of wavelet optical flow estimation and HLNLC classification can achieve more accurate and precise object detection.

Acknowledgments

This work was supported by the National Basic Research Program of China (973 Program) (2012CB821206), the National Natural Science Foundation of China (no. 91024001, no. 61070142) and the Beijing Natural Science Foundation (no. 4111002).