Abstract
The correlation filtering algorithm of infrared spectral data for dim and small target tracking is studied to improve the tracking accuracy of small and weak targets and to track small and weak targets in real time. After the image noise reduction processing by the mean shift filtering algorithm, the infrared small and weak target image data model is constructed by using the denoised infrared small and weak target image. And the brightness value and position of unknown small and weak targets are obtained. The tracking and measurement model of small and weak targets is built. The joint probabilistic data association algorithm is used to calculate the probability that each measurement is associated with its possible source targets, and the particle filter is used to update the tracking status of small and weak targets to achieve real-time tracking of small and weak targets. The experimental results show that the algorithm can enhance the edge contour information of small and weak images, so as to accurately track small and weak targets moving in any track, and has good real-time tracking and accuracy. There is a small deviation between the tracking track of weak and small targets tracked by the algorithm and the actual track, and the root mean square difference of tracking weak and small targets is within 2 pixels. In addition, the detection probability of detecting weak and small targets is less affected by SNR environmental factors.
1. Introduction
Infrared imaging is the process of converting the infrared radiation and background information of the target into infrared images [1]. Infrared imaging technology is a new type of high-tech that is very useful for both civil and military. It is widely used in optical remote sensing, night navigation, and target detection. Today, a large number of stealth aircraft and stealth radars have appeared. The limitations of the traditional detection and guidance tool, radar, are becoming more and more serious. However, infrared imaging has a strong antijamming capability and will not generate various radiation in the operation. It has good concealment and strong survivability. However, in the modern highly information-based and intelligent war, the infrared detection system must find and track dangerous targets from a long distance in order to win time for the decision-making of the command system and the response of the weapon system. Because the target needs to be found at a long distance, the target only occupies a few pixels, or even less than one pixel, in the imaging plane of the infrared detector, and is imaged as a weak infrared target [2]. Therefore, how to track and identify infrared weak and small targets has become a hotspot and difficulty in the field of infrared image.
Infrared target detection is very important for infrared search and tracking applications, such as early warning and precision guidance. The accurate and robust target extraction method can help people to determine the target area of interest in time and bring useful guidance and help to people’s life and production. Infrared target detection, especially for weak and small targets [3], was performed. The imaging area of the target in the infrared image is small, usually occupying only a few pixels, presenting point-like features and lacking texture information. Therefore, it is impossible to detect and track the target using traditional image processing technology. In infrared search and tracking, weak infrared targets are difficult to detect, especially under the interference of low illumination, unknown background clutter, and noise. On this basis, the detection and tracking of weak targets in infrared images is a hot topic at present. It is very difficult to realize the effective detection of weak and small targets in infrared video sequences through independent detection frame by frame. Generally, the mode of integration of detection and tracking is better. The basic idea of implementing visual tracking includes finding the target object in the video or image sequence, then determining its exact position in the next consecutive frame, and generating the corresponding trajectory of the object. How to improve the accuracy and success rate of the system is the current research focus. In recent years, the research of weak infrared target tracking has various new algorithms emerge in endlessly.
For long-range infrared imaging tracking, there is a difficult problem of stable tracking of small and weak targets. In long-distance target tracking, the target appears as a point, without shape and structure characteristics, and the intensity is weak. The point-like targets are in the complex sea and air background composed of clouds and waves and become weak small point targets in the complex environment [4]. The target point is easily submerged by noise [5]. Using a single image processing method, it is impossible to accurately track and detect the target and also to accurately grasp its moving track. Therefore, it is very valuable to track small and weak targets. At present, many scholars have done some exploration on the tracking algorithm for small and weak targets. For example, Chen et al. [6] proposed an efficient adaptive probability assumption density (PHD) filter. On this basis, a new method based on generalized newly generated target is proposed. This method uses the quantization information of the amplitude and phase of the newly generated target to effectively suppress the interference and clutter in MRFT, thus achieving effective tracking of weak targets. This detection method is mainly aimed at dense clutter, and it is prone to error in tracking and detection of small and weak targets. Zhan et al. [7] proposed a dynamic programming predetection tracking algorithm. The algorithm used an improved local region difference operator to extract the image features of weak and small targets, designed a new search strategy according to the attention transfer mechanism, updated the search range in real time through the moving speed of the target, and realized the tracking of weak and small targets. The attention transfer mechanism cannot accurately update the search range, and the tracking target will be lost, and the weak target cannot be accurately tracked. Bao et al. [8] proposed a filter based on multiple models (MM), which adds random variables representing the motion model to the target state, separates the target existence variables from the target state, calculates the probability of existence, and realizes joint detection and small and weak target tracking. The training of this network takes a long time, and different small and weak targets need to be trained separately. Rxa et al.’s [9] experiments show that the proposed algorithm can effectively improve the tracking accuracy and success rate of objects. Chen et al. [10] proposed a superresolution reconstruction of facial images using joint expression learning algorithm. On this basis, the focus block and the remaining block in the depth residual network are fused to obtain a face image that is very similar to the high-resolution face image and difficult to be recognized by the discriminator. Pang et al. [11] studied a remote sensing image feature extraction method based on temporal-spatial saliency for weak infrared targets in remote sensing images. Pang et al. [12] applied it to the detection of infrared weak targets on the basis of greedy bilateral decomposition. Due to the use of two greedy double factorization methods, the computational efficiency of this method has been significantly improved. These four algorithms have achieved good results in the detection and tracking of weak targets, which has certain reference significance for the research work of this paper.
On this basis, the infrared spectral image is processed and good image processing results are obtained. The main contributions of this paper are as follows: (1)In this paper, the multiframe image of weak and small targets is denoised, and the target is detected according to the continuity of target motion. At the same time, in the problem of target detection and signal detection in images, this paper introduces the basic theory of hypothesis detection, which improves the accuracy of target detection from infrared image sequences(2)In this paper, the joint probability data association algorithm and particle filter method are used to track the target in the image sequence. After obtaining each independent moving target, the target and its area are determined, and the real-time tracking of small and weak targets is realized
2. Correlation Filtering Algorithm of Infrared Spectral Data for Weak and Small Target Tracking
2.1. Image Noise Reduction Based on Mean Shift Filtering
Before tracking the small and weak target, the infrared camera is used to acquire the moving image. The process steps of acquiring the small and weak target image by the infrared camera are as follows: (1)Determination of temperature range: if infrared thermal imaging is used to measure temperature by different infrared radiation emitted by different objects, the temperature range must be determined before use; otherwise, the test structure may be inaccurate. This working principle is helpful to improve the working efficiency of the thermal imager and reduce the temperature measurement time(2)Adjustment of the focal length: the focal length is adjusted before use. If the range of the focal length is too high or too low, it is not conducive to reading the temperature. At present, most of the thermal imagers in the market have automatic focusing function and can be manually focused on this basis. Users can cooperate according to their own needs(3)Setting parameters: different environmental parameters are set according to different environments to make the thermal imaging detection effect more accurate
Because the small and weak targets in the acquired image are affected by noise and have no obvious shape. This paper proposes a mean shift filter algorithm to reduce noise in the image. Mean shift is a statistical clustering method. By converting image pixels into sample points in the feature space containing subject to mean shift clustering, so as to realize the operation of the weak and small target image space and reduce the noise of the image. The algorithm is completely dependent on the sample points in the feature space for analysis, does not need any prior knowledge, and has fast convergence speed. It is widely used in the field of weak and small target image processing [13–15].
Given samples in the -dimensional Euclidean space Gk, , at point , kernel function , and the positive definite wideband matrix are used, the kernel density of the density function at the defined point is estimated as where represents the coordinate of the observation point and represents the weight of the feature space sample , abbreviated as . The kernel function represents the similarity measure between sample and kernel center point . is the contour function of , , and the constant . The wideband matrix represents the influence range of the kernel function. And is the Mahalanobis distance. Because the gradient value of the density function is equal to its slope value,
where . Let
The equation is the expression equation of the mean shift vector. Let
Taking equations (5) and (6) into equation (4), respectively, it can obtain the following:
In the contour function set, any function can ensure that is always positive when applied and since the direction of gradient is always in the direction of increasing function value [16]. When the gradient is 0, the iteration ends, and the convergence point is the position of the local density maximum.
The mean shift algorithm is generally based on the following assumptions: (i)Hypothesis 1: the weight of each sample in the feature space is equal, i.e., (ii)Hypothesis 2: each sample in the feature space is distributed independently and simultaneously, and the structure in different directions of each sample is also the same. Generally, it is assumed that the bandwidth matrix is proportional to the identity matrix, that is,
The mean shift iteration equation under the assumption is as follows:
The mean shift algorithm is to obtain the convergence point of through iteration under the condition that the kernel function is selected and the allowable error is given, so as to find the local density center [17].
The gray space and the position space are independent of each other, and the multivariable kernel function can be decomposed into the form of the product of two spatial kernel functions:
where and and are the positions of the kernel function and the bandwidth of the gray space, respectively. The scale of and determines the extent of the estimation of the density gradient value at the -point; is the position component in the feature space, and is the gray level component; is the dimension of the gray space. The location and spatial bandwidth mainly affect the computing speed, which is determined by the visual task. The size ss of the gray space bandwidth is the resolution. The larger the bandwidth is, the more image details will be ignored.
2.2. Construction of Image Observation Model for Small and Weak Targets
Using the weak and small targets collected by the infrared camera, the observation model of the sequence images of the weak and small targets is constructed. The measurement model is as follows:
where represents the two-dimensional spatial coordinates of the image, represents the objective, represents a discrete-time sampling point, represents background clutter and short-time stationary process, represents zero mean and Gaussian-like noise, and o represents a random sample.
The weak and small target model is as follows:
where represents the total number of unknown targets at time and and denote the brightness value (short-time stationary) and position of the unknown target.
2.3. Tracking and Measurement Model of Infrared Weak and Small Targets
The target distance is long. It is assumed that the weak small target moves most rapidly in a short time [18]. The weak small target tracking model is expressed as follows:
where represents a state vector, is the position of the target at time , is the moving speed of the target at time , is the image sample interval, is the particle propagation radius, and is the zero mean white noise process at time .
The measurement model can use the observation equation [19] of particle filter theory, which can be described as follows:
where and are coordinate positions of the target at time .
2.4. Joint Probabilistic Data Association
Data association in multitarget tracking is a process in which the effective echo (output of tracking gate logic) is compared with the predicted track of the known target to determine the correct measurement track pairing. The definition of bar Sholom convergence is adopted, that is, in a set of target tracking gates, the intersection between any target tracking gate and at least one other target tracking gate is not empty. And the intersection between any target tracking gate that does not belong to this set and any target tracking gate in this set is empty. It is assumed that the multitarget tracking gate has been calculated based on a certain wave gate design theory with the current target state prediction value as the center, and the clustering relationship of the multitarget tracking gate has been calculated based on a certain algorithm [20–22].
PDA can track a single target very effectively, but it is easy to generate error tracking in the target dense environment. In this regard, a data association algorithm is proposed [23]. In order to express the complex relationship between effective echo and each target tracking gate, the concept of confirmation matrix [24] is proposed and defined:
where indicates whether the measurement falls within the confirmation gate of the target , indicates the number of measurements, and indicates that there is no target. At this time, the elements of the corresponding column are all 1, because any measurement may be caused by clutter or false alarm. When equivalency enters the tracking door, some metrics will be generated from multiple targets. The purpose of joint probability data correlation is to calculate the possibility that each observation value is associated with different possible source objects [25].
Using equations (10) and (11), the state update equation of JPDA is as follows:
where represents the state vector of the target at time , represents the state prediction vector of the target , and represents the joint innovation:
where represents the correlation probability between the target and the observation and represents the filtering innovation.
In terms of algorithm, JPDAF differs from PDAF only in the calculation of association probability . In JPDAF, since it considers the feasible joint events of all targets and tracks, the correlation probability can be expressed as follows:
where represents the observation equation, represents the number of feasible events and represents whether the observation originates from the target in the feasible event . If it originates from the target, it is equal to 1; otherwise, it is 0. In addition, these data also need to meet two preconditions: (1) Each data is unique. (2) For a specified object, at most one observation value can be used to use the object as a light source. At moment , the conditional probability of the joint event can be expressed as follows:
where is the target detection indicator and is the normalization constant, if the target is associated with the observation in . The value of is equal to 1; and otherwise, it is 0. is the number of false observation events, represents the detection probability of target , and represents the effective gate volume of the track.
2.5. Real-Time Tracking Algorithm of Weak and Small Multitargets Based on Data Association and Particle Filter
Particle filter is applied to joint probability data association, and particles are used to represent the edge distribution of each target, so that multitarget tracking under nonlinear non-Gaussian model can be processed. The flow of weak target tracking algorithm is as follows: let be the state of target at time predicted by the measurement data at time . The tracking algorithm process at time is as follows: (1)The position of the target at time is predicted by the state transition using the data at time :
i.e., the spatial propagation of particles, where represents the particle propagation radius (2)At time , the tracking window (the possible moving area of the target) is opened according to the dispersion degree of the particles, and then, the detection is carried out in the window to obtain the possible measurement value(3)For the observation and , the joint correlation probability between the measurement and the target is calculated(4)“New interest” is calculated, i.e.,(5)The weighted sum of “new interest” is calculated:(6)According to the innovation weighted sum, the weight of the particle support point set of the target is calculated, and the support point set is obtained(7)The target status value is updated, namely,(8)Resampling: samples are re-extracted from the sample set according to the weight of the samples to form a new support point set (9)The tracking system receives the next frame image and returns to step 1
3. Experimental Analysis
This project plans to select a set of weak aircraft detection and tracking dataset based on infrared images (https://sciencedb.cn/dataSet/handle/902). The acquired scene includes multiple scenes such as sky and ground, including multiple scenes, including 22 segments of data, 30 tracks, 16177 images, and 16944 objects. Each object corresponds to the tag location, and each data segment corresponds to the tag file. Using the aerial infrared image sequence of a region, the tracking of weak targets in the clouds is realized. The video sequence used is 600 pictures. This method is used to process the collected weak target and process it to test its denoising performance. And the original image, the reference [8] multimodel efficient particle filter, the reference [10] combination representation learning, and the mean-shift filtering method proposed in this paper are used for noise reduction. The original image is shown in Figure 1. The reference [8] multimodel efficient particle filter, the reference [10] combination representation learning, and the mean-shift filtering noise reduction results proposed in this paper are shown in Figures 2–4.
As shown in Figure 1, there are cloud interference and a large amount of noise in the original image. The flying target is very small and stored in the cloud. In Figure 2, the reference [8] multi-model efficient particle filter is used. At the same time, the image edge becomes blurred, the contrast of the image is reduced, and the image still has noise Figure 3. The combination of reference [10] shows that learning cannot suppress Gaussian noise well, some noise points exist, the target enhancement effect is not obvious, and the shape information of flying targets is fuzzy. The average migration algorithm described in Figure 4 can well preserve the boundary of the weak target in the image and enhance its contrast with the background under the condition of suppressing noise. The effectiveness of the algorithm is verified by simulation, which lays the foundation for the accurate tracking of small and weak targets.
The infrared image sequences are actually acquired, and the signal-to-noise ratios are 3-4, 4-5, and 5-6 db, respectively. Each infrared image sequence has 30 frames, and the image size is . The parameters of the algorithm are as follows: the number of particles is 1000, the number of iterations is 2000, and the threshold is half of the maximum gray value in the field of view. In the acquired infrared image sequence, the target enters the field of view of the infrared imager at frame 5 and disappears at frame 70. Simulation environment is as follows: simulation software MATLAB, desktop computer with main frequency of 2.11 GHz, and memory of 8 G. During the simulation experiment, the detected weak and small targets are marked with a red box with a window size of . The center of the box is the predicted position of the algorithm for the target, and the sliding step is 1. The original infrared image of a small and weak target and the tracking results of this method are shown in Figures 5(a)–5(c) and 6(a)–6(c).
(a) Frame 10
(b) Frame 20
(c) Frame30
(a) Frame 10
(b) Frame 20
(c) Frame 30
It is shown in Figures 5 and 6 that the algorithm in this paper tracks small and weak targets. When the sequence images are 10, 20, and 30 frames, the small and weak targets are detected and marked, and the dynamic tracking of small and weak targets can be realized.
Taking any small and weak target in Figure 6 as the experimental object, the small and weak target detection and tracking is carried out through the continuous video sequence. The number of targets in the small and weak target video sequence is 1, and the action trajectory of the experimental data can be observed through the small and weak target trajectory. The image signal-to-noise ratio and the detection rate of the dataset are selected as the basis, and the results of the small and weak target tracking experiment are evaluated by precision measurement and root mean square error, where the precision plot refers to the Euclidean distance between the center coordinate of the predicted position of the weak and small target and the center coordinate of the marked true value. It is calculated in pixels. The final result is expressed by the average precision measurement, that is, the square root of the ratio of the deviation square between the detection value of the weak and small target. The root mean square error of weak and small target tracking and detection is very sensitive to the mean deviation of a group of data and can well reflect the accuracy of the experiment. The calculation formula is shown in
where is the target observation value for the th time and is the observation number.
The position and tracking results of small and weak targets are shown in Figure 7, and the root mean square error of the tracking position of small and weak targets is shown in Figure 8.
It is shown in Figure 7 that there is a small gap between the estimated positioning and tracking results of small and weak targets and the actual positioning and tracking results, and the estimated motion trajectory of small and weak targets is basically consistent with the actual trajectory. It is shown in Figure 8 that the root mean square error of the tracking trajectory of small and weak targets is generally within 2 pixels. The experimental results show that there is a small deviation.
Aiming at the problems in weak target tracking, an improved weak target detection method is proposed. Under the comparison of various signal noise, the performance evaluation of this method is shown in Table 1.
As can be seen from Table 1, at 3-4 dB, the algorithm has a 99.38% probability of detecting weak targets, and the detection probability increases with the increase of the signal-to-noise ratio. In the case of 5 ~ 6 dB, the recognition rate of this method for weak targets reaches 100%. On this basis, a wavelet transform method based on wavelet transform is proposed. Practice has proved that this algorithm can effectively increase the detection probability of the system and can maintain good stability under large signal-noise ratio.
On this basis, this paper will track and detect multiple particles under the same signal-to-noise ratio and compare them to study their tracking and detection effects on multiple particles. Table 2 shows the comparison of detection performance.
It is shown in Table 2 that the tracking and detection effects of the proposed method for weak and weak targets are different under the same signal-to-noise ratio. With the increase of the number of particles, the detection probability and tracking accuracy of this method will also improve, but in each frame, this method needs a longer processing time. The effectiveness of this method in weak target detection is verified by simulation. The number of particles is proportional to the time required to detect and track weaker objects. When using this method, you need to set appropriate parameters for particles.
The target tracking time is taken as an indicator, and the infrared weak and small targets are tracked by using the combination of multiple model efficient particle filter in reference [8], the combination of representation learning in reference [10], and the mean shift filtering method proposed in this paper. The tracking time of different methods is shown in Table 3.
It is shown in Table 3 that the tracking time of the three methods is increasing. Compared with the three algorithm, it has a faster tracking speed for weak infrared targets, up to 20.4 milliseconds. Through the analysis of infrared weak target, it is proved that the average shift filter can effectively improve the tracking effect of infrared weak target.
4. Discussion
Weak and small target tracking is applied in industry, agriculture, medicine, transportation, aerospace, and other fields. Effective tracking of infrared weak and small target can provide necessary guidance and help for production and operation. Scholars and researchers have proposed a variety. However, small infrared targets in complex background are easily submerged. During the movement of the target, the possible changes in illumination, the occlusion of the target, or the temporary loss of the target increase the tracking difficulty. Therefore, it is still challenging to detect and track small infrared targets.
This paper mainly studies the influence of infrared image’s own weakness, complex background, gray level change, and other factors on the image and gives the corresponding algorithm. The mean shift filter is proposed to reduce the noise of weak and small targets and improve the definition of the outline of weak and small targets. JPDA algorithm can update the real-time tracking of small and weak targets. The results show that the mean shift filter can not only enhance the target but also suppress the background noise. The particle filter tracking method in this paper tracks the same sequence of images, and the tracking error rate is lower and the center position error is smaller when the instantaneous acceleration of the target is large and the movement direction changes are complex. Its robustness and accuracy are more worthy of affirmation.
5. Conclusion
Infrared target detection and tracking technology is of great significance. Because the infrared target tracking technology involves many kinds of algorithms and the automatic recognition system is complex, the author’s theoretical level needs to be further improved. The algorithm introduced in this paper and some simulation experiments also have some limitations. Therefore, the research work of this paper is not perfect. Future research needs to be further expanded and deepened, mainly in the following aspects: (1)Infrared small target detection using human eye contrast mechanism can accurately detect the position of the target even under extremely low SNR conditions, and there is a problem that the algorithm takes a long time. In the next step, the method of adding preliminary detection after image preprocessing can be considered to lock the approximate region of interest and only calculate the local contrast for this region(2)At present, the algorithm of small target tracking using human eye contrast and particle filter, as well as the algorithm of combining learning memory and particle filter, are faced with the problem of long computing time, which needs to be further studied and solved(3)Considering that in the process of infrared tracking, there may be large changes in target scale and motion attitude. Or multitarget tracking is required. Therefore, the tracking accuracy and performance of this method still have the possibility of further improvement. That is, in the follow-up work, we can carry out targeted simulation tests for the above situations to verify whether the proposed algorithm can be applied to various scenarios and improve the problems in the test
Data Availability
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Conflicts of Interest
The authors declare that they have no competing interests.