Spatial Object Tracking Using an Enhanced Mean Shift Method Based on Perceptual Spatial-Space Generation Model

Abstract

Object tracking is one of the fundamental problems in computer vision, but existing efficient methods may not be suitable for spatial object tracking. Therefore, it is necessary to propose a more intelligent mathematical model. In this paper, we present an intelligent modeling method using an enhanced mean shift method based on a perceptual spatial-space generation model. We use a series of basic and composite graphic operators to complete signal perceptual transformation. The Monte Carlo contour detection method could overcome the dimensions problem of existing local filters. We also propose the enhanced mean shift method with estimation of spatial shape parameters. This method could adaptively adjust tracking areas and eliminate spatial background interference. Extensive experiments on a variety of spatial video sequences with comparison to several state-of-the-art methods demonstrate that our method could achieve reliable and accurate spatial object tracking.

1. Introduction

Mathematical formalism is probably the most precise and logical language in science research. It is typical for researchers in pure natural sciences to attempt to describe observed phenomena using mathematical correlations. However, because of real-world scenarios, it is often very difficult to construct a perfect and permanent mathematical model for one specific issue in engineering fields [1]. During the last years, the effort concentrated in self-optimizing and self-adaption was leading to a new field between mathematics and applications, called intelligent modeling [2]. In this paper, we propose a new intelligent modeling method using the enhanced mean shift method based on a perceptual spatial-space generation model for spatial object tracking. Object tracking has been applied to many fields, such as video surveillance [3], robot recognition [4], and traffic control [5]. In spatial on-orbit docking, object tracking could be used to track spatial aircraft and assist with ground control. Because the spatial images are mainly generated from low-rate videos [6] or airborne spectral imagery [7], which are captured by the aircraft sensors [8, 9], their resolution and spatial-temporal coverage are not very ideal. In addition, because of differences in the sensor spectral bands, acquisition position, and contrast gradient setting, there are shifts in the relative position and scale zoom in multisource images with the same scene. All this will bring influence to spatial object tracking results.

Recently, multidimensional decomposition and multiscale representation methods have been widely applied to image processing and computer vision. Mumford and Gidas proposed the stochastic model [10], in which truncation errors and noise interference could be isolated from the discrete domain. Witkin and Koenderink proposed the image scale-space model [11, 12], which cleared noise interference in fine scales, and analysis errors could decrease in coarse scales. Burt and Lindeberg proposed the coarse-to-fine model [13, 14], which could reduce useless gradients in gradient entropy calculation. In the object tracking field, the traditional method is rectangular block region tagging [15, 16]. In [17], Isard and Blake applied a local filter to object tracking field. Sun and Liu proposed using a combination of the local description and global representation in object tracking [18]. Recently, a graphics model based on a Bayesian neural network was also applied to continuous object tracking [19]. However, if we applied these methods to spatial object tracking, spatial background clutter and moving object overlapping appeared in different scenes. The existing multiscale deviation will seriously reduce the spatial object tracking accuracy.

In this paper, we propose a spatial object tracking method using an enhanced mean shift method based on a perceptual spatial-space generation model. We detect the spatial object continuity and saliency between different scales in the perceptual spatial-space generation model. The enhanced mean shift considers the relevance between motion area and static background. It could achieve more robust object tracking. Our proposed method is shown in Figure 1. This paper is organized as follows. Section 2 describes the perceptual spatial-space generation model. Section 3 proposes an enhanced mean shift method. Section 4 shows experimental results. Section 5 is the conclusion.

Figure 1 

Enhanced mean shift method based on a perceptual spatial-space generation model.

2. Perceptual Spatial-Space Generation Model

Spatial images have a high amount of data structure, and their processing may not obey existing image processing model assumptions. In this paper, we propose a perceptual spatial-space generation model. It consists of two parts: a prototype pyramid, labeled 𝑆[0,𝑛]=(𝑆₀,…,𝑆_𝑛), and a set of perceptual transform rules, labeled 𝑅[0,𝑛−1]=(𝑅₀,𝑅₁,…,𝑅_𝑛−1). The set 𝐼[0,𝑛] is the Gaussian transform pyramid. Our goal is to set a common variable 𝑝(𝐼[0,𝑛],𝑆[0,𝑛],𝑅[0,𝑛−1]) and a maximized posterior probability 𝑝(𝑆[0,𝑛],𝑅[0,𝑛−1]∣𝐼[0,𝑛]) that can be used to calculate the priority prototype pyramid and the perceptual transform rules.

2.1. Prototype Pyramid Generation

Generation model is a joint probability function with prototype 𝑆 and image 𝐼, and Δ_𝑅 is a dictionary which includes image primitives, such as blobs, edges, crosses, and bars. It can be expressed as 𝑝𝐼;𝑆;Δ_𝑅=𝑝𝐼∣𝑆;Δ_𝑅𝑝(𝑆). (1)

The decomposition probability could be divided into primitives and texture.

Consider the following: 𝑝𝐼∣𝑆;Δ_𝑅=^𝑁𝑅_𝑘=1exp−_{()𝑢,𝑣∈𝑅}𝐼(𝑢,𝑣)−𝑅_𝑘(𝑢,𝑣)²2𝜎²_𝑜⋅^𝑁𝑅_𝑗=1𝑝𝐼_𝑅,𝑗∣𝐼_𝑅;𝜎_𝑗, (2)

where 𝑆=⟨𝑉,𝐸⟩ is the properties graphics, 𝑉 is the collection of primitives in 𝑆, (𝑢,𝑣) denoting the primitives in the dictionary Δ, and 𝜎 is the variance of the corresponding primitive features. The priority model 𝑝(𝑆) is an uneven Gibbs model, which defines the graphical attributes on 𝑆. It focuses on continuity properties in the perceptual model, such as smoothness, continuity, and typical functions.

Consider the following: 𝑝(𝑆)=exp𝜀⁴_{𝑑=0()𝑢,𝑣∈𝑅}𝑅_𝑑𝜑(𝑢,𝑣)+(1−𝜀)_{()𝑢,𝑣∈𝑅}𝜑𝐵_𝑢,𝐵_𝑣, (3)

where 𝑅_𝑑 is the primitive mark in 𝑆 and its connection degree is 𝑑. 𝜑(𝐵_𝑢,𝐵_𝑣) is potential association for two correlation functions. Because there is uncertainty in the inner perception posterior probability, the prototype pyramid may not appear continuous for each layer calculation. In order to ensure transition and consistency of single frame diagram, we define a set of graphical operation factors 𝑅_𝑘=𝑟_𝑘,1,𝑟_𝑘,2,…,𝑟_{𝑘,𝑚(𝑘)},𝑟_𝑘,𝑖∈_gram. (4)

Graphical operation factors could synthesize detected graphic edges into pairs of characteristic bridges. Each bridge is related with the properties of probability function. The conversion from 𝑆_𝑘 to 𝑆_𝑘+1 is realized by a series of conversion rules 𝑅_𝑘, and rule order could directly determine conversion efficiency. The generative rule graphic path from 𝑆_𝑘 to 𝑆_𝑘+1could be expressed as 𝑝𝑅_𝑘=𝑝𝑆_𝑘+1∣𝑆_𝑘=^𝑚(𝑘)_𝑖=1𝑝𝑟_𝑘,𝑖+1𝑚^𝑖_𝑘^𝑚𝑖𝑘_𝑗=1𝑤_𝑘^𝑖,𝑗−𝑑_𝑗−𝑑^𝑖_𝑘,𝑝(𝐼[][][=0,𝑛,𝑆0,𝑛,𝑅0,𝑛−1])_𝑛𝑘=0𝑝𝐼_𝑘∣𝑆_𝑘;Δ_𝑠𝑘⋅𝑝𝑆₀⋅^𝑛−1_𝑘=0^𝑚(𝑘)_𝑗=1𝑝𝑟_𝑘,𝑖, (5)

where 𝑆 is the optimal-calculated prototype. Under the condition that there will be no loss in perception model accuracy, we assume that 𝑆_𝑠𝑚 begins to decay from 𝑆 through the single operation factor. 𝐼_𝑠𝑚 will gradually reduce the resolution, and 𝑆_𝑠𝑚 will also have a related complexity. The posterior probability could be expressed as 𝑃()𝑝𝐼𝑠𝑚=log^𝑠𝑚∣𝑆_𝑠𝑚𝑝𝐼_𝑠𝑚∣𝑆+𝜆_𝑠𝑚𝑝𝑆log^𝑠𝑚𝑝(𝑆)𝑝𝑆=log^𝑠𝑚∣𝐼_𝑠𝑚𝑝𝑆∣𝐼_𝑠𝑚. (6)

A layered reduced perception model will not completely adapt to the complex model 𝑆, so the first logarithmic ratio is often negative. The parameter 𝜆 is used to balance model fitness and complexity. If 𝜆=1, we could launch simplified 𝑃(𝑠𝑚). 𝜆_𝑠𝑚 could be decided in the following range: 𝑝𝐼0<log^𝑠𝑚∣𝑆_𝑠𝑚𝑝𝐼_𝑠𝑚∣𝑆+𝜆_𝑠𝑚𝑝𝑆log^𝑠𝑚𝑝(𝑆)<1. (7)

The transform between 𝑆_𝑘 and 𝑆_𝑘+1is achieved using a group of greed detection. The accurate scale of graphical operation factor will make differences based on the subjective goal. We suppose that the graphical operation factor is between 𝐼 and 𝐼_𝑠𝑚𝑎₁𝑝𝐼=−log^𝑠𝑚∣𝑆_𝑠𝑚𝑝𝐼_𝑠𝑚∣𝑆,𝑏₁=𝜆_𝑠𝑚𝑝𝑆log^𝑠𝑚𝑝(𝑆),𝑎₂𝑝=−log𝐼∣𝑆_𝑠𝑚𝑝(𝐼∣𝑆),𝑏₂=𝜆_𝑠𝑚𝑝𝑆log^𝑠𝑚𝑝(𝑆). (8)

Based on formulas (6) and (7), we determined that the corresponding interval is 𝑁𝑟_𝑎−𝑟_𝑘^𝑖,𝑗;0,𝜎²_𝑎<𝜆_𝑠𝑚<𝑁𝑟_𝑏−𝑟_𝑏^𝑖,𝑗;0,𝜎²_𝑏. (9)

2.2. Perceptual Transform of Prototype Pyramid

In this section, our goal is to determine the optimum conversion path and deduce the hidden graphics prototype. Our method is scanning the prototype pyramid from top to bottom based on each primitive learning decision rule. Our method can be divided into three steps.

Step 1 (prototype pyramid independent calculation). We apply a pyramid algorithm in the bottom of image 𝐼 and calculate the Gaussian pyramid 𝑆₀. Because each prototype layer is calculated using a MAP estimation [20], there is a certain loss in the continuity of the prototype pyramid. The specific formula is as follows: (𝑆[0,𝑛],𝑅[0,𝑛−1])=argmax^𝑛_𝑘=0𝑝𝐼_𝑘∣𝑆_𝑘;Δ_𝑘⋅𝑝𝑆₀^𝑛_𝑘=1^𝑚(𝑘)_𝑗=1𝑝𝜆_𝑘,𝑗. (10)

Step 2 (pattern matching from bottom to up). We match the image prototype attribute from 𝑆_𝑘 to 𝑆_𝑘+1 using an image registration algorithm from a previous study [21]. We use 𝑥, 𝑦, 𝑧, and 𝑒 as a judgment function to process each node 𝑖 and obtain the related image characteristics at each time. Specifically, the matching degree between the 𝑖th node at the 𝑘th scale and the 𝑗th node at the (𝑘+1)th scale can be expressed as =1match𝑖,𝑗𝑍−𝑥(𝑖)𝑗exp−𝑥²2𝜎²_𝑥−𝑦(𝑖)−𝑦𝑗²2𝜎²_𝑦−𝑧(𝑖)−𝑧𝑗²2𝜎²_𝑧−𝑒(𝑖)−𝑒𝑗²2𝜎²_𝑒, (11)
where 𝜎 is the variance of related features. For similarity matching between 𝑆_𝑘 and 𝑆_𝑘+1, this formulation allows the empty variable prototype to appear in 𝑆_𝑘. We multiply 𝑆_𝑘 by the related variance 𝜎 and obtain a homologous 𝑆_𝑘+1 with subsidiary value. Pattern matching results will be used as the initial parameter for the Markov chain matching in the next step.
Consider the following: 𝑃𝑆_𝑘,𝑆_𝑘+1=𝑥;𝑦;𝑧;𝑒⟵𝑆²∶𝑘^′_min<𝑘<𝑘^′_min×(𝑥;𝑧)∈match𝑣_𝑖(𝑘),𝑣_𝑖(𝑘+1). (12)

Step 3 (Markov chain matching). Due to the uncertainty for the initial perception in the posterior probability model and the relative complexity for dynamic graphic structure mining, we use the reversibility of Markov chain matching to match a perceptual transform. The Markov chain includes 25 pairs of reversible jump. In each path of Markov chain matching, there is a reversible jump between 𝑋₁ and 𝑋₂ [22]. These reversible jumps are related to their corresponding grammars, and each pair of these rules is based on probability selection. We use this mechanism to optimize the perception conversion path, which could lead a cross-scale continuous perception prediction.
Consider the following: (𝑆[0,𝑛],Markov[0,𝑛−1])=arg/max^𝑛_𝑘=0𝑝(𝐼𝑘∣𝑆𝑘;Δ𝑅)⋅^𝑛_𝑖^𝑚_𝑗𝑔𝑟_𝑖,𝑗. (13)

2.3. Object Contour Evolution

The problem of such matching strategy is that it does not contain any matching, which could split a long contour into short edges. We propose a new Monte Carlo contour detection method. This method mainly chooses the right standard in certain scale using spatial-space domain knowledge. Our proposed method uses the weight set {𝑥_𝑖,𝑡,𝑤_𝑖,𝑡} to estimate the posterior probability density 𝑝(𝑥_𝑖,𝑡,𝑤_𝑖,𝑡). According to the resampling theory [23], it is feasible to calculate the sample specimen 𝑄(𝑥,𝑤) with appropriate weights from the normal density distribution 𝑋^𝑖_𝑡.

Consider the following: 𝑤_𝑡^𝑖,𝑛=𝑝𝑧^𝑖_𝑡∣𝑥^𝑖_𝑡,𝑧_𝑡^𝑁(𝑖),𝑥^𝑖_0∶𝑡−1,𝑧^𝑖_1∶𝑡−1,𝑍^𝑁(𝑖)_1∶𝑡−1𝑝𝑧^𝑖_𝑡,𝑍^𝑁(𝑖)_1∶𝑡−1∣𝑧^𝑖_1∶𝑡−1().⋅𝑄𝑥,𝑤 (14)

For the sequence sample, the important probability 𝑄(𝑥,𝑤) can be chosen using the following mode: 𝑄()𝑥𝑥,𝑤=𝑞^𝑖_𝑡∣𝑥^𝑖_0∶𝑡−1,𝑧^𝑖_1∶𝑡,𝑍^𝑁(𝑖)_1∶𝑡𝑞𝑥^𝑖_0∶𝑡−1∣𝑥^𝑖_1∶𝑡−1,𝑧^𝑖_1∶𝑡,𝑍^𝑁(𝑖)_1∶𝑡−1. (15)

The entire probability can be approximated with a simplified formulation 𝑤_𝑡^𝑖,𝑛=𝑝𝑥^𝑖_𝑡∣𝑧^𝑖_𝑡𝑝𝑥_𝑡^𝑖,𝑛∣𝑧^𝑖,𝑛_𝑡−1𝑞𝑥^𝑖_𝑡∣𝑥^𝑖_0∶𝑡−1,𝑧^𝑖_1∶𝑡,𝑍^𝑁(𝑖)_1∶𝑡×_{(𝑖)𝑗∈𝑁}^𝑁_𝑡=1𝑝𝑧^𝑖_𝑡∣𝑥_𝑡^𝑗,𝑛𝑝𝑥_𝑡^𝑗,𝑛∣𝑧_𝑡^𝑗,𝑛. (16)

We use only one part of the whole sample; 𝑝(𝑥_𝑡^𝑖,𝑛∣𝑧^𝑖,𝑛_𝑡−1) simulates the interaction value between the two adjacent areas 𝑥_𝑡^𝑖,𝑛 and 𝑥_𝑡^𝑗,𝑙. The local probability 𝑝(𝑧^𝑗_𝑡∣𝑥_𝑡^𝑗,𝑙) is the weight of the interactive area. In this paper, we use a sequence of Monte Carlo simulations to estimate the interactive part 𝑝(𝑥^𝑗_𝑡∣𝑥^𝑖_𝑡) with its related estimation weight. The density importance function can be defined as follows: 𝑞𝑥^𝑖_0∶𝑡∣𝑥^𝑖_𝑡,𝑧^𝑖_1∶𝑡,𝑍^𝑁(𝑖)_1∶𝑡,Z^𝑅(𝐼)_1∶𝑡=𝑞𝑥^𝑖_𝑡∣𝑥^𝑖_0∶𝑡−1,𝑧^𝑖_1∶𝑡,𝑍^𝑁(𝑖)_1∶𝑡,Z^𝑅(𝐼)_1∶𝑡×𝑞𝑥^𝑖_0∶𝑡−1∣𝑥^𝑖_1∶𝑡,𝑧^𝑖_1∶𝑡,𝑍^𝑁(𝑖)_1∶𝑡−1,Z^𝑅(𝐼)_1∶𝑡−1. (17)

Based on the resampling theory, the sampling weight can be updated as follows: 𝑤_𝑡^𝑖,𝑛=𝑤^𝑖,𝑛_𝑡−1𝑝𝑥^𝑖_𝑡∣𝑧^𝑖_𝑡𝑝𝑥_𝑡^𝑖,𝑛∣𝑧^𝑖,𝑛_𝑡−1𝑞𝑥^𝑖_𝑡∣𝑥^𝑖_0∶𝑡−1,𝑧^𝑖_1∶𝑡,𝑍^𝑁(𝑖)_1∶𝑡,𝑍^𝑅(𝐼)_1∶𝑡×_{(𝑖)𝑗∈𝑁}^𝑁_𝑡=1𝑝𝑧^𝑖_𝑡∣𝑥_𝑡^𝑗,𝑛𝑝𝑥_𝑡^𝑗,𝑛∣𝑧_𝑡^𝑗,𝑛×_{(𝑖)𝑗∈𝑁}^𝑁𝐿_𝐿=1𝑝𝑧^𝐿_𝑡∣𝑥_𝑡^𝑘,𝐿𝑝𝑥_𝑡^𝑘,𝐿∣𝑧_𝑡^𝐿,𝑛, (18)

where 𝑁 is the sample retrieval in the 𝑖th part, 𝐿 is the sample length of the 𝑗th part, and 𝑧^𝐿_𝑡 is the sample set of the 𝑗th part. As we use the Markov property, the density function 𝑝(𝑧^𝑘_𝑡∣𝑥_𝑡^𝑘,𝐿) could do further approximation relied on the unit product of local observation similarity unit 𝐾.

Consider the following: 𝑝𝑧^𝑘_𝑡∣𝑥_𝑡^𝑘,𝐿=𝑝𝑧_𝑡^𝑁(𝑘),𝑧_𝑡^𝑅(𝐿)∣𝑥^𝑘_𝑡,𝑥^𝐿_0∶𝑡−1=𝑝𝑧^𝑘_𝑡∣𝑥^𝑘_𝑡⋅𝑝𝑥^𝑘_𝑡,𝑍_𝑡^𝑁(𝑘),𝑍_𝑡^𝑅(𝐿)∣𝑥^𝑘_0∶𝑡−1𝑝𝑧^𝑘_𝑡,𝑍_𝑡^𝑁(𝑘)∣𝑍^𝑘_1∶𝑡−1,𝑍^𝑁(𝑡)_1∶𝑡−1,𝑍^𝑅(𝐿)_1∶𝑡−1. (19)

In our proposed framework, all parts of the detected object will be tracked at the same time. There is no need to calculate all unit probabilities 𝑝(𝑧^𝑘_𝑡∣𝑥_𝑡^𝑘,𝐿), so we can use the local possibility and directly estimate the weight of all units.

2.4. Markov Random Field Representation

We propose Markov random field representation for perception generation model. We define the pixel perception set 𝑋, in which any two arbitrary pixels are adjacent. The adjacent relation is an interactive relationship; in the an adjacent pixels system 𝐸={𝛿(𝑥,𝑦)∶(𝑥,𝑦)∈𝐼}, if (𝑥,𝑦) is adjacent point of (𝑤,𝑣), then (𝑤,𝑣) is also adjacent with (𝑥,𝑦), and 𝑝(𝐼) is the Markov random field with respect to the perception set 𝑋𝑝𝐼_𝑥,𝑦∣𝑋𝑥,𝑦=𝑝𝐼_𝑥,𝑦∣𝐼𝛿⧵𝑋𝑥,𝑦. (20)

We can define 𝐼(𝛿) as the pixels set that contains all pixels of 𝑋. The Markov property mentioned in formula (20) relies on the density distribution of the adjacent pixels. According to S. Geman and D. Geman [24], the Markov random field-related pixels in system 𝑋 can be rewritten as the following Gibbs distribution: 𝑝(𝐼)=12𝑍Σ_𝐶^1/2exp−_𝑋𝐺_𝑥(𝐼(𝐸))⋅⁻¹_𝐶𝐸^𝑖_𝑥−𝐸^𝑖_𝑦, (21)

where 𝐺_𝑥(𝐼(𝐸)) is the potential variable function defined in 𝐼(𝛿). 𝑍 is a conventional constant, which can maintain the 𝑝(𝐼) sum up to 1. For spatial object tracking, the perception latent variables that appear in pairs are difficult to describe accurately. If 𝑋 has an amount of pixels, then 𝐺_𝑥 will be a multidimensional function. Here, we use the topological model to solve the above problems. Assuming that the spatial perceptual rules set is 𝜑(𝐼), then 𝜑_𝑥,𝑦(𝐼) could extract the local characteristics near pixels (𝑥,𝑦). A specific example is 𝜑_𝑥,𝑦(𝐼)=⟨𝐼,𝛽_{𝑥,𝑦,𝑠}⟩, 𝑠=(𝐼,𝜃). We transform the image 𝐼 with the Gibbs distribution =1𝐺𝐷𝐼∣𝛽𝑍exp𝐺𝐷𝑝𝐼^(𝑘)−𝐺𝐷^′𝑝(𝑋=1)𝑍exp^𝑆_{𝑠=1𝑥,𝑦}𝛽_{𝑥,𝑦,𝑠}𝑋_{𝑥,𝑦,𝑠}(𝐼)−_𝑥,𝑦𝑝(𝐼)𝑝_{𝑥,𝑦,𝑠}⋅(𝐼),𝐼−𝑝 (22)

where 𝛽_{𝑥,𝑦,𝑠} is the low-dimensional image characteristic function set, 𝑍 is the conventional constant that depends on 𝛽, and 𝑋_{𝑥,𝑦,𝑠}=(𝐼,𝛽_{𝑥,𝑦,𝑠}). Assuming that the normal distribution is 𝑝(𝐼), 𝑝_{𝑥,𝑦,𝑠} is the normal distribution of 𝛽_{𝑥,𝑦,𝑠} under 𝐼−𝑝(𝐼). 𝛽 could guarantee 𝑓^∗(𝐼)=𝑓(𝐼|𝛽). For each (𝑥,𝑦,𝑠), the marginal distribution of 𝜑(𝑥,𝑦,𝑠) is 𝑝(𝑥,𝑦,𝑠). With any given 𝑓=𝑓(𝐼|𝛽),𝐷(𝑝|𝑓^∗)≤𝐷(𝑝|𝑓), the specific calculation of the axiom is as follows: 𝐷𝑝∣𝑓−𝐷𝑝∣𝑓^∗𝑍𝛽=log𝑍𝛽^∗𝑆_{𝑠=1𝑥,𝑦}𝛽^∗_{𝑥,𝑦,𝑠}𝑋_{𝑥,𝑦,𝑘}(𝐼)−𝛽_{𝑥,𝑦,𝑠}𝑋_{𝑥,𝑦,𝑘}(𝐼)=𝐷𝑓^∗∣∣𝑓≥0. (23)

In the formulation of 𝑓(𝐼|𝛽),𝑓^∗ can be seen as the best approximation of probability 𝑝, and it also can be marked as the “maximum likelihood.” From the minimized 𝐷(𝑝||𝑓(𝐼|𝛽)) to the maximized 𝐸[log𝑓(𝐼|𝛽)], the marginal probability 𝜑(𝑥,𝑦,𝑠) can be 𝑝_{𝑥,𝑦,𝑠}() for any (𝑥,𝑦,𝑠) under 𝑝(𝐼). So 𝐻(𝑓^∗)−𝐻(𝑓)=𝐷(𝑝||𝑓^∗).

Consider the following: 𝐻𝑓^∗−𝐻𝑓=𝐸_𝑞log𝑞(𝐼)−𝐸_𝑓^∗log𝑓^∗(𝐼)=𝐸_𝑞log𝑞(𝐼)−𝐸_𝑞log𝑓^∗(𝐼𝑞𝑓)=𝐷||^∗. (24)

In order to model the observed image, we assume that 𝛽_{𝑥,𝑦,𝑠}()=𝛽_𝑠(), 𝛽 does not depend on (𝑥,𝑦). We can continue to parameterize the 𝛽_𝑠 or normalize 𝛽 in a low-dimensional scale. If we normalize 𝛽_𝑠 in 𝑆=1,…,𝐿 and make 𝛽_𝑠(𝜑_{𝑥,𝑦,𝑠}(𝐼))=𝛽_𝑠𝑙, we could rewrite the Gibbs distribution as follows: =1𝐺𝐷𝐼∣𝛽𝑍exp^𝑆_{𝑠=1𝑥,𝑦𝑙}𝛽_{𝑥,𝑦,𝑠}𝑋_{𝑥,𝑦,𝑠}(𝐼)−_{𝑥,𝑦𝑙}𝑝(𝐼)𝑝⋅(𝐼)=1𝑥,𝑦,𝑠𝐼−𝑝𝑍exp_𝑠𝑙𝛽_𝑠𝑙𝐻_𝑘𝑙(𝐼)−_{𝑥,𝑦𝑙}𝑝^′(𝐼)⋅𝐻_𝑘(𝐼=1)𝑍exp_𝑠𝛽_𝑘,𝐻_𝑘(𝐼), (25)

where 𝐻_𝑘(𝐼)=𝑝_𝑥,𝑦𝜑_{𝑥,𝑦,𝑠}(𝐼) represents the quantity of the effective points 𝜑_{𝑥,𝑦,𝑠}(𝐼) which falls into the interval 𝑆 and 𝐻_𝑘=(𝐻_𝑘,𝐿) is the marginal matrix related to {𝜑_{𝑥,𝑦,𝑠},𝑆}. If we want to find the maximum estimate coefficient 𝛽, we need to calculate the spatial-scale statistical coefficient 𝐻_𝑘(𝐼)𝐸_𝛽𝐻_𝑘(𝐼=1)𝑛^𝑛_𝑘=1𝐻𝑝𝐼^(𝑘)−𝐻_𝑘𝐼_observed. (26)

In other words, we need to match the spatial data and the related model. The most suitable model is determined by 𝐻_𝑘(𝐼_observed), and 𝐸_𝛽[𝐻_𝑘(𝐼)] is an average parameter; value 𝛽 is a natural parameter. In the perceptual model, there will be a global balance variable. We can make a minor adjustment 𝑝𝐼₀∣𝐼_𝜌0=1𝑍exp_{𝑘𝑥,𝑦∈𝑝0}𝛽_𝑘Φ_{𝑥,𝑦,𝑠}+_𝑘𝐸_𝑤𝑝log𝑤∣𝐼_𝑠𝑝(),𝑤∣𝐹 (27)

where 𝐼_𝜌0 is the approximation value of global observation estimated from observed spatial images. The local equilibrium parameter defined in any pixel will obey the specific distribution 𝑝(𝐼)=1𝑍exp^𝐾_𝑘=1𝐶_𝑚,𝑘𝐵_𝑘+𝜀_𝑚−𝜂_𝑗𝐹_𝑗(𝐼)𝐵_𝑘+𝜆^𝐾_𝑘=1𝑆𝐶_𝑚,𝑘, (28)

where 𝜂_𝑗 is local approximation for pixel 𝑝₀. It is the only distribution that has similar effects with perceptual model.

3. Enhanced Mean Shift Method

3.1. Mean Shift

In this section, we propose an enhanced mean shift method. Our method uses the mathematical recursion method [25]. The spatial tracking object will be represented by a spatial histogram consisting of a weighted evaluation, in which the probability estimation function 𝑝(𝑦) and 𝑞(𝑥₀) are used to represent the potential motion probability in images 𝐼(𝑦) and 𝐼(𝑥₀). The histogram variables can be expressed as follows: 𝑝𝑦=𝑐|Σ|²_𝑖𝑘𝑦_𝑖𝑦^𝑇_𝑖𝛿𝑏_𝑢𝐼𝑦_𝑖+𝑐−𝑢2|Σ|^1/2𝑤_𝑗,𝑘𝑦^𝑇_𝑗⁻¹𝑦_𝑗,𝑞𝑥₀=𝑐|Σ|²_𝑗𝑘𝑥_𝑗𝑥^𝑇_𝑗𝛿𝑏_𝑢𝐼𝑥_𝑗+𝑐−𝑢2|Σ|^1/2𝑤_𝑗,𝑘𝑦^𝑇_𝑗⁻¹𝑦_𝑗, (29)

where 𝑦_𝑖=(𝑦_𝑇𝑖−𝑦), 𝑥_𝑗=(𝑥_𝑇𝑗−𝑥), and Σ are the representatives of weight function, 𝑏_𝑢(𝐼(𝑦_𝑖)) and 𝑏_𝑢(𝐼(𝑦_𝑗)) are motion estimations for positions 𝑦_𝑖 and 𝑦_𝑗, 𝑤_𝑗 is the box parameter, 𝐶 is normalized constant, 𝑢=1,…,𝑚, 𝑚 contains all the binary pixels, and 𝑦 and 𝑥 are the center of related kernel function. The Bhattacharyya coefficient 𝜌 will be used to detect similarity between tracking object area and the potential background.

Consider the following: 𝜌=∑𝑝,𝑞_𝑢𝑝_𝑦𝑥,𝑦,Σ𝑞_𝑥𝐿_𝑥(𝑖)×𝐿_𝑦𝑗𝐿min^𝑦𝑗𝐿_𝑥(𝑖,𝐿)^𝑥(𝑖)𝐿_𝑦𝑗. (30)

We apply the first-order Taylor sequence extension, in which (𝑥,𝑦) are the coordinates of the center position in the previous frame, and then we obtain the following extended formulas: 𝜌=_𝑢12𝑝_𝑦𝑞_𝑥+𝑐𝑥,𝑦,Σ2|Σ|²_𝑗𝜔_𝑗𝑘𝑦^𝑇_𝑗𝑥^𝑇_𝑖,𝜔_𝑗=_𝑢𝑝_𝑦𝑞_𝑥𝛿𝑏_𝑢𝐼𝑦_𝑗−𝑢. (31)

The center of kernel function can be determined by the estimate of 𝜌(𝑥,𝑦)𝛿=𝛼₁^𝑛_𝑖=1𝜌𝑥^𝑖_𝑝,𝑥^𝑖+𝛼₂^𝑛_𝑖=2𝜌𝑥^𝑖_𝑝,𝑥^𝑙_𝑝−𝜌𝑥^𝑖,𝑥^𝑙. (32)

In order to estimate the kernel function, the normalized bandwidth will be applied to a similarity judgment. The normalized bandwidth can be obtained by estimating |Σ|𝜌(𝑥,𝑦)𝜌=2𝑥,𝑦∑1−𝑟^𝑛𝑗=1𝜔_𝑗𝑥_𝑗𝑦^𝑇_𝑗∑⁻¹𝑥^𝑇_𝑖𝑔∑^𝑛_𝑗=1𝜔_𝑗𝑘𝑦^𝑇_𝑗∑⁻¹𝑥^𝑇_𝑖+1𝑁_pre^𝑁pre_𝑖=1max_𝑗𝐷^𝑖_pre,𝐷^𝑗_cur, (33)

where 𝑦^𝑇_𝑗=(𝑦_𝑗−𝑦) could accurately determine 𝜌(𝑥,𝑦). Equations (32) and (33) will be calculated in an alternative iteration until the estimated parameters can cover all the variables.

3.2. Estimation of Spatial Shape Parameters

We also use the iterated function to determine boundary parameter 𝑉_𝑇⁽²⁾, which contains five fully adjustable affine box parameters. These parameters are the width, height, length, orientation, and center location. The orientation 𝜃 will be defined as the angle between the horizontal and width matrix. The box with width 𝑤 and height ℎ is the ellipse area between the long and short coordinate system. The relationship between 𝜃, ℎ, 𝑤 and the bandwidth matrix Σ can be expressed as follows: Σ=𝑅^𝑇(𝜃)ℎ2²00𝜔2²𝑅(𝜃),where𝑅=.cos𝜃−sin𝜃sin𝜃cos𝜃 (34)

These parameters can be calculated using the octave decomposition method [26]; the specific formula is as follows:𝑃𝑓^pre_𝑡∣𝑓^cur_𝑡=𝑁𝑓^pre_𝑡∶𝑓^cur_𝑡,diag𝜎²_𝑤,𝜎²_ℎ,𝜎²_𝑙,𝜎²_𝑜,𝜎²_𝑐,if𝑓^cur_𝑡𝑃isnotnull,_𝑣1,if𝑓^cur_𝑡isnull, (35)

where 𝑓^pre_𝑡 and 𝑓^cur_𝑡are the octave decomposition components of the previous frame and the current frame.

3.3. Spatial Object Tracking Using Enhanced Mean Shift

In order to limit the possibility that background pixels appear in the tracking object, we use a relatively small elliptical area, in which the contract domain is determined by the factor 𝐾; in our experiments, 𝐾=0.7. The elliptical area can be defined as follows: 𝐸𝐴=1−^𝑀_𝑖=1𝑢𝑝_𝑢^𝑖,(𝑗)𝑦,Σ𝑞_𝑢^𝑖,(𝑗)+12^𝑀_𝑖=1𝐶_𝑀Σ_𝑖^𝑁1/2_𝑗=1𝑤^𝑣_𝑗𝑘𝑦^𝑇_𝑗⁻¹_𝑣𝑦_𝑗. (36)

In our proposed method, we should determine whether the previous frame motion area (Object 𝐴) will be used to guide the next frame object (Object 𝐵) tracking. If the number of continuous characteristic pixels and the Bhattacharyya coefficient for 𝐴 are both higher, the initial rectangular tracking area for 𝐵 will refer to 𝐴’s settings, and the tracking area will be determined by the previous frame mean shift.

Consider the following: 𝑉_𝑡⁽²⁾=𝑉_𝑡⁽¹⁾,ifdist_𝑡=∑⁴_𝑘=1𝑥_𝑡,𝑘−𝑥_{𝑡−1,𝑘}²<𝑇₁⁽²⁾,𝑉⁽²⁾_𝑡−1,ifdist_𝑡=∑⁴_𝑘=1𝑥_𝑡+1,𝑘−𝑥_𝑡,𝑘²<𝑇₂⁽²⁾, (37)

where 𝑇₁⁽²⁾ and 𝑇₂⁽²⁾determine the threshold value. In order to solve drift and error propagation, we use enhanced mean shift for frame resampling. The object tracking resampling operation can be summarized as follows: 𝜌≈^𝑀_𝑖=1𝑢12𝑝^𝑖_𝑢𝑞^𝑖_𝑢𝑥⁽²⁾_𝑡,𝑖−𝑥⁽²⁾_{𝑡−1,𝑖}+^𝑀_{𝑖=1𝑦𝑡∈𝑅𝑡}𝑅^(obj)_𝑡−12Σ_𝑖^1/2𝑤^𝑖_𝑡𝑉^(obj)_𝑡−1𝑦^𝑇_𝑗⁻¹_𝑖𝑦_𝑗,where𝑅_𝑡⟵𝑅^(obj)_𝑡−1,𝑉_𝑡⟵𝑉^(obj)_𝑡−1,𝜌_𝑡⟵0, (38)

where 𝑥⁽²⁾_𝑡,𝐼 and 𝑥⁽²⁾_{𝑡−1,𝐼} are the four-dimensional motion areas in frames 𝑡 and 𝑡−1 and 𝑇₃⁽²⁾and 𝑇₄⁽²⁾ are the threshold values determined by the graphic distance and the similarity shape.

4. Experimental Result

We conducted experiments on four different spatial video sequences, in which tracking objects are spatial satellites and aircrafts. We uniformed the sequence image size for the same spatial resolution; each frame is 320×256. The superiority of our proposed algorithm will be validated by an intuitive performance and objective evaluation.

4.1. Contour Evolution in Prototype Pyramid

Contour evolution experimental results are shown in Figure 2. Each prototype pyramid layer is calculated independently. The performances have shown that the contour evolution has a better continuity in the layer-by-layer prototype pyramid, and the approximate effects are derived from a perceptual spatial-space generation model and are closer to the human visual perception system.

(a)

(b)

(c)

(d)

Figure 2 

Contour evolution on spatial video sequences. (a) Original video frame and (b)–(d) contour evolution from a coarse to fine scale.

4.2. Markov Random Field Representation

The Markov random field representation is shown in Figure 3. The significant representative region derivate from the perceptual spatial-space generation model contains significant feature information, which is closely related to the different color and texture distribution in the spatial area. The experimental results show that the region representation effect, which has been smoothed, could highlight the sensitivity of the motion area more than the initial representation.

(a)

(b)

(c)

Figure 3 

Markov random field representation in the perceptual spatial-space generation model. (a) Original frame and (b)–(c) initial and global smoothing representation.

4.3. Enhanced Mean Shift Object Tracking

We conducted experiments on the ten video sequences, in which the tracking objects include spatial satellite and aircraft, highway and park surveillance, and the human body. The enhanced mean shift conducted an iterative calculation 20 times on each video sequence. The normalized matrix bandwidth parameters are determined by a different experimental sample. In our experiments, it is 0.63 for video sequences 1, 2, and 3, 0.42 for video sequences 4 and 6, 0.56 for video sequences 5 and 8, 0.21 for video sequence 7, and 0.60 for video sequence 9 and 10.

4.3.1. Satellite-1, 2, and 3

Tracker-1: the particle filter will have a negative impact on the horizontal direction, and the motion estimation will appear noncontinuous. Tracker-2: the distance metric learning will affect motion area determination, and the rectangular window tracking will produce some deviation. Our proposed algorithm can better track the spatial objects, and satellites and aircraft can be completely contained in the rectangular window with a similar color distribution and background quiver. The edge deviation variance can be controlled well using spatial object detection, with no offset and blurring.

4.3.2. Automobile-1 and 2

In a nighttime environment, as the weak light and brightness, Tracker-1 and Tracker-2 obtain a vague tracking result, and the confusion area between background and the tracking object becomes larger. In the Automobile-2 sequence in particular, as the illumination from other automotive foreground lamps, Track-1 produces particularly serious deviation. Our proposed enhanced mean shift method could distinguish the tracking object from background confusion, and the tracking results do not appear to obviously deviate.

4.3.3. Highway

Tracker-1: this method will lose the tracking center in some frames. There are some cross-rectangular windows between the far and close vision sequences. Shape errors exist in the rectangle window estimations. Tracker-2: the tracking results also have a tracking deviation in the far vision. Our proposed method can maintain consistency between different visions and does not appear to have huge deviations.

4.3.4. Running, Walking

There are no obvious differences between the different methods. Except for some slow motion (walking body), Tracker-2 shows some partial deviations.

4.3.5. Automobile-3, Walking-2

These cases are used to further test the robustness of our proposed method in the scene containing two or more moving objects. We could observe that Tracker-2 results in less accurate boxes probably due to its poor estimate of different moving object center position in the same scene. The performance of Tracker-1 is somewhat better; however, it also produces partial deviations. Our proposed method is more robust when dealing with scenarios of two or more moving objects.

4.4. Objective Evaluation

We use three objective evaluations for evaluating the object tracking performance (Figure 4).

Figure 4 

Experimental results from the tracking, marked by the bounding box. The red (solid line) box is from our proposed method, the blue (dashed line) box is Tracker-1 from the particle filters in [27], and the green (dashed line) box is Tracker-2 from the online distance metric learning in [28]. The rows 1–10 contain images from the videos “Satellite-1,” “Satellite-2,” “Satellite-3,” “Automobile-1,” “Running,” “Automobile-2,” “Highway,” “Walking,” “Automobile-3” and “Walking-2,” respectively.

4.4.1. Euclidian Distance

The Euclidian distance is the distance between rectangular windows obtained by tracking methods and the artificially marked. The specific calculation is as follows: 1𝐷=4⁴_𝑖=1𝑥_𝐴,𝑖−𝑥_{𝐺𝑇,𝑖}²+𝑦_𝐴,𝑖−𝑦_{𝐺𝑇,𝑖}², (39)

where (𝑥_𝐴,𝑖,𝑦_𝐴,𝑖), (𝑥_{𝐺𝑇,𝑖},𝑦_{𝐺𝑇,𝑖}), and 𝑖=1,2,3,4 are corner coordinates of the rectangular window calculated by tracking methods and artificially marked. Figure 5 shows the Euclidian distance between the tracked and artificially marked areas for our proposed method and the two other trackers on the videos.

Figure 5 

Euclidian distances between the tracked and artificially marked areas. Red curve: our proposed method; blue curve: Tracker-1 in [27]; green curve: Tracker-2 in [28].

The averaged Euclidian distance calculated on all videos is shown in Table 1. Compared to the distance values from Tracker-1 and Tracker-2, our proposed method clearly shows smaller and bounded Euclidian distances for the tested videos.

Table 1 

Results of the averaged Euclidian distance over all frames in each video.

4.4.2. Mean Square Errors (MSEs)

We have 1MSE=𝑁^𝑁_𝑖=1𝑥^𝐴_𝑖−𝑥_𝑖^𝐺𝑇2+𝑦^𝐴_𝑖−𝑦_𝑖^𝐺𝑇2, (40)

where (𝑥^𝐴_𝑖,𝑦^𝐴_𝑖), (𝑥_𝑖^𝐺𝑇,𝑦_𝑖^𝐺𝑇) is the center of the tracking area obtained by methods and artificially marked, respectively. 𝑁 is the total number of video sequence frames. The experimental results are shown in Table 2 and, it can be seen that our proposed method has the minimum MSE, which means it has the lowest tracking deviation. Our method has obvious advantages compared to other methods.

Table 2 

Results of averaged MSE errors for the tracked box from our proposed method, Tracker-1, and Tracker-2.

4.4.3. Bhattacharyya Distance

The Bhattacharyya distance is used to judge the deviation degree between the tracking area and the actual motion area. The specific calculation method is as follows: mean_𝐴 and mean_𝐺𝑇 are the mean vectors with respect to the tracking area and are calculated by our method and artificially marked. The variables cov_𝐴 and cov_𝐺𝑇 are covariance matrices with respect to the tracking area and are calculated by our method and artificially marked.

Consider the following: 1BD=8mean_𝐴−mean_𝐺𝑇^𝑇cov_𝐴−cov_𝐺𝑇2⁻¹×mean_𝐴−mean_𝐺𝑇+12lncov_𝐴+cov_𝐺𝑇/2cov_𝐴^1/2cov_𝐺𝑇^1/2. (41)

The Bhattacharyya distance between the tracked object area and the artificial marked region is shown in Figure 6. Among nine case studies, our proposed method has shown a marked improvement on the tracking accuracy as compared with the two existing trackers (Tracker-1 and Tracker-2). It is mainly due to our combination of perceptual spatial-space generation model and the enhanced mean shift. The averaged Bhattacharyya distance on different video is shown in Table 3. Our proposed method has the smallest average tracking deviation between different methods.

Table 3 

Averaged Bhattacharyya distances for our proposed method, Tracker-1, and Tracker-2.

Figure 6 

Bhattacharyya distances between the tracking method and the artificial markers. Red curve: our proposed method; blue curve: Tracker-1 in [27]; green curve: Tracker-2 in [28].

5. Conclusions

In this paper, we propose a new intelligent modeling method using the enhanced mean shift method based on a perceptual spatial-space model for spatial object tracking. The perceptual spatial-space model can obtain a continuous spatial object contour and highlight tracking object saliency. Enhanced mean shift method uses enhanced version mean shift, which focuses on the estimation of spatial shape parameters. The method could effectively cope with severe spatial interferences. The comparison between our method and other state-of-the-art methods demonstrates that our proposed method has a higher tracking accuracy and precision. In future research, we can incorporate more spatial object information, such as spatial textures and aircraft shapes, into our intelligent model to generate a more robust spatial object tracking method.

Acknowledgments

This work was supported by the National Basic Research Program of China (973 Program) (2012CB821206), the National Natural Science Foundation of China (no. 91024001 and no. 61070142), and the Beijing Natural Science Foundation (no. 4111002).