Abstract
Infrared sensing technology can be well used for night observation, which is becoming an important measure for battlefield reconnaissance. It is a powerful way to implement precision strikes and situational awareness by improving the ability of target recognition based on infrared images. For the problem of infrared image recognition, the Light Gradient Boosting Machine (LightGBM) is employed to select the outline descriptors extracted based on the elliptic Fourier series (EFS), which is combined with sparse representation-based classification (SRC) to achieve target recognition. First, based on the target outlines in the infrared image, the multi-order outline descriptors are extracted to characterize the essential characteristics of the target to be recognized. Then, the LightGBM feature selection algorithm is used to screen the multi-order outline descriptors to reduce redundancy and improve the pertinence of features. Finally, the selected outline descriptors are classified based on SRC. The method effectively improves the effectiveness of the final features through the feature selection of LightGBM and reduces the computational complexity of classification at the same time, which is beneficial to improve the overall recognition performance. The mid-wave infrared (MWIR) dataset of various targets is employed to carry out verification experiments for the proposed method under three different conditions of original samples, noisy samples, and partially occluded samples. By comparing the proposed method with several types of existing infrared target recognition methods, the results show that the proposed method can achieve better performance.
1. Introduction
The development and maturity of infrared imaging technologies provides an important tool for night observation [1–4]. In the civil field, traffic monitoring can be performed based on infrared images or videos, which effectively avoids accidents. In the military field, infrared imaging can be used for battlefield monitoring and precision strikes. The target recognition method based on infrared image aims to confirm the target category in the measured samples, so as to obtain valuable information for technical support. As a supervised classification problem, infrared image target recognition usually adopts a two-stage process of feature extraction and classification. Feature extraction acquires features such as intensity distribution, local texture, key points, and regional distributions of the target through the analysis of infrared images. In [5], the boundary features were extracted for target recognition. In [6], the image moments were employed to analyze the regional characteristics for discriminating different categories of targets. Local textual descriptors, i.e., histogram of oriented gradient (HOG) features, were applied to infrared image target recognition in [7]. The multi-scale HOG features were developed in [8] to describe the intensity distribution properties of infrared images for target recognition. The classification process employs mature classifiers to train the extracted features and obtain the corresponding categories of the unknown samples. Typical classifiers used in infrared image target recognition include support vector machine (SVM) [8], sparse representation-based classification (SRC) [9], neural networks [10], etc. Since the emergence of deep learning theory and algorithms, the classification models represented by convolutional neural network (CNN) have been widely used in image processing and have become a powerful tool for infrared target recognition [10–13].
The results of feature extraction are used as the input of subsequent decision making. In this sense, the effectiveness of the extracted features directly determines the validity of the final decision. According to the existing literatures, most of the features used in infrared target recognition were directly applied based on mature algorithms in the field of pattern recognition, lacking targeted analysis and screening. In fact, some of the extracted features may have little contribution to the final decision or even cause negative influences on the classification process. For example, the redundant features would definitely increase the computational load and possibly cause some wrong decisions. In [14, 15], the genetic algorithm and particle swarm optimization algorithm were used to select the extracted Zernike moment features for synthetic aperture radar (SAR) target recognition. A basic comparison between the results of using all the features and the selected features verified the necessity and validity of feature selection. Inspired by this work, this paper employs the Light Gradient Boosting Machine (LightGBM) [16–20] for the selection of outline descriptors extracted by elliptic Fourier series (EFS) [21–23] and applies it in infrared image target recognition. As a new boosting algorithm, LightGBM is operated on the gradient boosting decision tree (GBDT) developed in the early stage and optimizes learning through additive models and forward step-by-step algorithms. According to relevant research studies, LightGBM can effectively improve the learning speed and model accuracy. Based on LightGBM, the extracted multi-order outline descriptors can be analyzed and screened. Only the effective components are maintained, and redundant ones are eliminated, so as to ensure the effectiveness of the final features. In the classification stage, SRC is used as the basic classifier to characterize and recognize the final selected outline descriptors. As a classifier based on the compressive sensing theory, SRC could adapt to different kinds of features and achieve robust performance [24–31]. In the experiments, the public mid-wave infrared (MWIR) target image dataset is employed to test and verify the proposed method, and the results reflect its effectiveness and robustness.
2. Outline Descriptors
The ellipse Fourier descriptor uses ellipse superposition to approach the boundary of a specific object. It can approach the boundary infinitely in the form of multiple harmonics and has the advantages of scale transformation, rotation transformation, and invariance of starting point transformation [21–23]. This paper first uses the method in [32–35] to obtain the outline point set of the target. For a closed boundary , it can be expressed as a vector form , where . Therefore, is the periodic function of the parameter , which can be expressed in the form of Fourier series as shown in the following equation:where , represents the EFS outlier descriptors, which are calculated as follows:
In equation (2), is the 0-order elliptic Fourier descriptor, which represents the DC component of the harmonic. It denotes the center of the closed boundary, which contains two parameters . For subharmonics, there are four coefficients . This paper adopts the first 6 orders of the EFS outline descriptors for the following classification.
3. Proposed Target Recognition Method
3.1. Feature Selection Based on LightGBM
The multi-layer features from the same images are usually not completely uncorrelated, and the contribution of different features to the target classification is also distinguished. As the number of extracted features increases, the amount of repetitive redundancy will increase, and the noise and error will also increase. The increase will not only increase the calculation amount of the algorithm but also reduce the recognition rate of the target. Therefore, it is necessary to select features that are beneficial to target classification. In this paper, LightGBM is used for feature selection [16–20], the training samples are put into LightGBM for training, the importance of each feature is calculated, and the key features are selected according to the importance to reduce the number of features required for classification. The classification performance reduces the recognition time at the same time.
The LightGBM algorithm measures the importance of feature attributes according to the total number of times the feature is used to split in all decision trees, sorts the feature elements in descending order, starts from the full set of sample features, and judges whether to exclude according to the accuracy of the result. The current feature with the lowest degree of importance is looped in this way to achieve feature selection. The specific process is described as follows (Algorithm 1).
| ||||||||||||||||
It can be seen that the proposed LightGBM feature selection algorithm sorts according to the importance of features and preliminary evaluation of the model results, which greatly reduces the volatility of features. It is beneficial to the improvement of the accuracy of target recognition in the later stage. At the same time, because the features of relatively low importance are deleted, the redundancy of the features is reduced, so the time consumption required for target recognition can also be reduced.
3.2. SRC for Classification
SRC is a non-parametric classifier developed from the compressive sensing theory, which was first applied to face recognition with excellent performance. In that work, the effectiveness and robustness of SRC were validated. Afterwards, SRC was widely used in pattern recognition like SAR target recognition [25–27]. For a sample from the class, SRC assumes that it can be linearly represented by the samples from the same class as follows:where is a test sample assumed from the class; comprises the training samples of the class; denotes the coefficient vector with only a few non-zero elements; and is the reconstruction error.
When the test sample is from an unknown class, SRC is performed on the global dictionary as follows:where denotes the coefficient vector over the global dictionary.
The optimization task in (4) can be robustly solved by the -norm relaxation or greedy algorithm like orthogonal matching pursuit (OMP). With an optimal solution of , the reconstruction errors related to different classes can be computed as equation (5) for decision making.where corresponds to the coefficients of training class.
3.3. Implementation Details
Figure 1 shows the basic process of the proposed method, which can be defined as two stages of feature extraction and classification decision according to traditional idea. In the feature extraction stage, the first step is to use EFS outline descriptors to analyze the target contour in the infrared image to obtain multi-order features. On this basis, LightGBM is used for training and learning, the best feature subset is selected, and the redundant components are eliminated. For the training samples, the same operation is followed to construct the corresponding global dictionary, which contains the outline descriptors corresponding to all the samples. Finally, SRC is used to classify the outline descriptors of the test sample and obtain its corresponding target category.
4. Experiments and Discussion
4.1. Dataset
The experiments are carried out on the MWIR dataset, which contains images of 10 typical military targets acquired by the mid-wave infrared sensors. Figure 2 shows the exemplar images of the 10 targets. According to the relevant experimental settings in [21], 120 images are randomly selected as the training set and 100 images are used as the test sample. During the experiment, four types of methods are selected for comparison, which are denoted as SVM, SRC, EFS, and CNN methods. Among them, the EFS method does not use the feature selection by LightGBM in the proposed method and directly classifies the extracted multi-order outline descriptors based on SRC.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
4.2. Original Test Samples
First, experiments are conducted on the basis of the original training and test samples. At this time, the test sample and the training sample come from similar conditions and their difference is small. So, the recognition problem is relatively simple. Table 1 shows the detailed recognition results of the ten types of targets in this paper. The recognition rates of all types of targets are above 97%, and the final average recognition rate is 97.9%, reflecting the effectiveness of the proposed method. Table 2 further compares the average recognition rates of different methods on the original samples. The method in this paper has the highest performance and shows its advantages. The performance of the CNN method under the current conditions is second only to the method in this paper, indicating the effectiveness of the deep learning model. Compared with the method that directly uses outline descriptors, this paper introduces LightGBM to analyze and select them, and the retained features further improve the recognition performance, which shows the effectiveness of the proposed method.
4.3. Noisy Test Samples
Natural noise, perceived interference, and sensor thermal noise may all cause the signal-to-noise ratio (SNR) of the final infrared image to be reduced. Therefore, it is necessary to improve the performance of the identification method under noise interference conditions. This paper firstly generates test sets with different signal-to-noise ratios based on the original test samples by adding white noise and then tests various methods to obtain the results shown in Figure 3. It can be seen that noise corruption has caused the performance of various methods to decrease, but the proposed method maintains the strongest robustness. Compared with SVM, the SRC method has performance advantages and reflects the adaptability of the sparse representation mechanism to the influence of noise. Despite the lack of secondary feature selection, the EFS method can still maintain good noise robustness, reflecting the stability of outline descriptors under noise interference. In detail, EFS outline descriptors are mainly related to the target contour, and the overall regional characteristics of the target can be kept robust under noise corruption conditions, so this type of method can achieve stable performance under noise conditions. The proposed method combines the advantages of outline descriptors, LightGBM-based feature selection, and sparse representation, so the final recognition result has advantages.
4.4. Partially Occluded Test Samples
There may be two reasons to cause partially occluded infrared images. One is that the target itself is partially absence, and the other is that it is partially occluded during the imaging process. In both cases, there are some missing target characteristics in the final image, which makes the recognition problem more complicated. In this paper, based on the original test samples, the partial occlusion algorithm is used to simulate the construction of test sets under different missing levels (measured by the ratio of the occluded area to the target area). Figure 4 shows the average recognition rate curve of various methods under partial missing conditions. It can be seen that the performance of all methods has been significantly reduced under the partial missing condition, and the method in this paper can maintain the highest recognition rate at all levels, indicating its robustness. The SRC method has a performance advantage over the SVM method, which shows the effectiveness of the sparse representation mechanism for partial deletions. The EFS method is at a disadvantage compared to the proposed method due to the lack of secondary feature screening, but it has certain advantages over other methods. This shows that the outline descriptors have certain adaptability to the damage of the target area caused by partial missing. In summary, this method combines outline descriptors, LightGBM-based feature screening, and SRC-based decision making to jointly improve the recognition performance under the current test conditions.
5. Conclusion
This paper proposes an infrared image target recognition method based on feature selection of EFS outline descriptors. LightGBM is used to screen the multi-order outline descriptors of infrared targets to eliminate redundant components. On this basis, SRC is used to classify the selected outline descriptors to determine the target category of the test sample. The proposed method improves the final recognition performance through effective extraction, secondary feature selection, and SRC-based decision making. Experiments are carried out on the MWIR dataset to classify 10 typical military targets. According to the experimental results, the following conclusions can be drawn. (1) The proposed method achieves better recognition performance on the original test samples, noisy samples, and partially occluded samples than the four types of comparison methods, showing its performance advantages. (2) The introduction of LightGBM for the selection of the original multi-order outline descriptors effectively improved the overall recognition performance in the proposed method. (3) The LightGBM-based feature selection can be regarded as a general framework for other types of features for different kinds of pattern recognition problems.
Data Availability
The dataset can be accessed upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.