Abstract

Aiming at the problem of fusion algorithm performance evaluation in multiradar information fusion system, firstly the hierarchical attribute model of track relevance performance evaluation model is established based on the structural model and functional model and quantization methods of evaluation indicators are given; secondly a combination weighting method is proposed to determine the weights of evaluation indicators, in which the objective and subjective weights are separately determined by criteria importance through intercriteria correlation (CRITIC) and trapezoidal fuzzy scale analytic hierarchy process (AHP), and then experience factor is introduced to obtain the combination weight; at last the improved technique for order preference by similarity to ideal solution (TOPSIS) replacing Euclidean distance with Kullback-Leibler divergence (KLD) is used to sort the weighted indicator value of the evaluation object. An example is given to illustrate the correctness and feasibility of the proposed method.

1. Introduction

Cyber-physical system (CPS) was listed as an important research project in USA competitiveness plan released by American Academy of Sciences in February 2006. CPS can be understood as a high-performance network intelligent information system based on embedded devices [1–8]. Through the highly integration and interaction of a series of calculation units and physical objects under network environment, the capacity of information processing, real-time communication, precise remote control, and component self-coordination is improved. So it is a hybrid autonomous system which is heterogeneous and multidimensional in time and space.

As an application of CPS in military field, command, control, communication, and intelligence (C³I) system has been developed and improved all over the world to complete specific strategic and tactical missions in nearly 20 years. How we analyze and process a large amount of information collected by various ways to make the right decisions in C³I system has become a new and important research branch of data processing [3, 5–9]. Multiradar information fusion system (mRIFS) is a typical example of C³I system. In mRIFS, a multilevel and multifaceted processing for the multiple radars data is executed, and the measurement information provided by a number of radars is analyzed, processed, and integrated to produce the desired synthesis information [10–12].

Through the information integration, the expanded time coverage area, and the increased confidence level, ambiguity can be reduced and spatial resolution and reliability can be improved. A number of distinctive algorithms of information fusion technology have been formed and developed. The K-nearest neighbor (KNN) algorithm is widely used in information fusion and other related respects [13]. Hayat and Khan analyzed the performance of various machine learning mechanisms for discriminating outer membrane proteins (OMPs) such as genetic programming, -nearest neighbor, and fuzzy -nearest neighbor (fuzzy K-NN) in conjunction with discrete methods such as amino acid composition, amphiphilic pseudo amino acid composition, split amino acid composition (SAAC), and hybrid versions of these methods and observed that fuzzy K-NN using SAAC based-features makes it quite effective in discriminating OMPs after the simulation [14]. Zhang and Wang investigated the combined feedback-feedforward tracking control problem for networked control systems (NCSs) under the discrete-time framework and proposed the controller design method by solving a nonlinear trace minimization problem. Finally, they gave an example on the control of a helicopter to illustrate the proposed design approach [15]. Tomašev et al. went a step further by embracing the soft approach and proposed several fuzzy measures for -nearest neighbor classification, all based on hubness, which express fuzziness of elements appearing in -neighborhoods of other points [16]. Zhang et al. investigated the vehicle lateral dynamics stabilization problem to enhance vehicle handling by considering time-varying longitudinal velocity. They described the longitudinal velocity by a polytope with finite vertices and proposed a novel technique to reduce the number of vertices [17]. Ran and Zhang studied a data fusion algorithm based on track fuzzy membership. Carlo simulation shows that the proposed algorithm is superior to weighted track association algorithm, and tracking system can obtain the target fusion track that has the same tracking performance of simple fusion (SF) algorithm [18]. Moosavian et al. surveyed a new scheme for fault diagnosis of main journal-bearings of internal combustion (IC) engine based on power spectral density (PSD) technique and two classifiers, namely, -nearest neighbor (KNN) and artificial neural network (ANN) [19]. Zhang et al. proposed a mode-dependent estimator with a fast sampling frequency such that the estimation can track the signal to be estimated with the nonuniformly sampled outputs. They obtained the corresponding stochastic estimation error system by using an augmentation technique and derived the linear-matrix-inequality-based sufficient conditions by studying the stochastic stability and the energy-to-peak performance of the estimation error system [20]. Zhang and Wang investigated H∞ the filtering problem of discrete-time Takagi-Sugeno (T-S) fuzzy systems in a network environment and presented a tunnel-diode circuit in a network environment to show the effectiveness and the advantage of the proposed design approach [21]. Huang and Wu proposed an algorithm of weighted covariance for centralized asynchronous fusion based on Kalman (WCCAF), which made linear minimum mean square error (LMMSE) the criterion, and considered the relevance of measurement noise [22]. Zhang et al. exploited the sliding mode control problem for a fluid power electrohydraulic actuator (EHA) system [23]. Zhang et al. investigated the problem of robust static output feedback (SOF) control for networked control systems (NCSs) subject to network-induced delays and missing data [24].

For the very extensive fusion object and different application requirement of all military departments, the demand for comprehensive decision-making approach for information fusion algorithms’ performance evaluation is ubiquitous [25]. The experiments for a variety of fusion algorithms and evaluation of the experimental results have a considerable significance to find the most suitable fusion algorithm for the current data source condition. Through the fusion algorithms’ performance evaluation, the processing capability of fusion algorithm under different conditions of data source can be shown by the evaluation results.

2. Multiradar Information Fusion System: An Application of Cyber-Physical Systems in Military C³I Field

A typical air traffic control (ATC) distributed mRIFS has a structural model as is shown in Figure 1. All radar stations complete the tracking process of detected target locally. After converting the original point track to local single radar track, the data will be sent to ATC control center through network to form multiradar track of all targets and will be sent to front display to provide a variety of control applications [7–9]. The flight plan, line, path, chart and meteorological information, and other related flight data are stored in system database. Additionally, the single radar data and multiradar fusion data are put in storage real time.

Figure 1 

The structural model of distributed mRIFS.

The most authoritative functional model of data fusion systems (DFS) is proposed by USA, and its simplified form is shown in Figure 2 with C³I system as an example. According to the function modules supported by comprehensive information process, the whole process is divided into two phases: in the first phase, integrate the data providing target state parameter estimation and classification information and enrich and update the database; in the second phase, based on the first phase and supported by the knowledge base, comprehensively infer the semantic information [4, 12]. The information fusion of mRIFS functional model is generally divided into five levels, namely, detection-level fusion, location-level fusion, attribute-level fusion, situation evaluation, and threat assessment.

Figure 2 

DFS functional model.

3. Hierarchical Attribute Model

Comprehensive decision-making is a conversion process which changes the multiple attributes system from high-dimensional to low-dimensional, in which it is required that decision-making results can reflect the information of original indicators as much as possible. Aiming at the actual needs of military electronic information system software’s development, we carry out the research of comprehensive decision-making approach for information fusion algorithms’ performance evaluation. On the basis of the analysis of performance indicators for mRIFS fusion algorithm, the hierarchical attribute model (HAM) of track relevance performance is built.

3.1. The Structure of Hierarchical Attribute Model

As is shown in Figure 3, the target of fusion algorithm performance evaluation (FAPE) can be divided into three standards, namely, data source quality standard (DSQ), track quality standard (TQ), and track association quality standard (TAQ). In this paper, we only analyze and study the eight indicators of TAQ, namely, non-common target ratio (nCTR), state position estimation error (SPEE), state speed estimation error (SSEE), correct relevant rate (CRR), error relevant rate (ERR), leakage relevant rate (LRR), correct separation rate (CSR), and track repeat factor (TRF). nCTR, SPEE, and SSEE belong to the substandard of local node information consistency (LNIC); CRR, ERR, and LRR belong to the substandard of relevant rate (RR); CSR and TRF belong to the substandard of fusion degree (FD).

Figure 3 

Track relevance performance evaluation model.

3.2. Indicator Definition and Its Quantization

Assuming that the number of integration centers and radars is 1 and , respectively, the outcome track number of the radars is . In the case of two radars and , their outcome track sets are and , which can form relevant information pairs; is the element number of the sets . We can get the following relationship equations: , , and , in which indicates the set of relevant information pairs composed by the same target; indicates the set of information pairs correctly judged to be relevant by relevant processing and indicates the set of information pairs correctly judged to be irrelevant by relevant processing; indicates the set of relevant information pairs composed by different targets; indicates the set of information pairs correctly judged to be irrelevant by relevant processing and indicates the set of information pairs mistakenly judged to be relevant by relevant processing.

Definition 1. This includes non-common target ratio (nCTR). The nCTR of two radars and can be represented by .

Definition 2. This include state position estimation error (SPEE) and state speed estimation error (SSEE). We assume that and are the longitude and latitude, while and are the speed in longitude and the speed in latitude, so the track state estimation vector can be represented as .
For the track in , its SPEE and SSEE are
If there are a couple of radars and , ,  .

Definition 3. This includes correct relevant rate (CRR). It means the rate of information pairs which are formed by the same objective and correctly judged to be relevant, .

Definition 4. This includes error relevant rate (ERR). It means the rate of information pairs which are formed by different objectives and mistakenly judged to be relevant, .

Assuming that the track , we get the following.(i)If part of is relevant to the true track of the objective and the other is falsely relevant, it means partial error relevance. The partial error relevance rate is , in which is the set of partial error relevant tracks.(ii)If part of is relevant to the true track of the objective and the other is relevant to the true track of the objective , , it means partial error transfer. The partial error transfer rate is , in which is the set of partial error transfer tracks.(iii)If the first half of is relevant to the true track of the objective and the second half is relevant to the true track of the objective , , it means error switch. The error switch rate is , in which is the set of error switch tracks.

Definition 5. This includes leakage relevant rate (LRR). It means the rate of information pairs which are formed by the same objective and mistakenly judged to be irrelevant, .

Definition 6. This includes correct separation rate (CSR). It means the rate of information pairs which are formed by different objectives and correctly judged to be irrelevant, .

Definition 7. This includes track repeat factor (TRF). If the number of tracks outcomes by integration center is and the number of true tracks is , the TRF is .

4. Comprehensive Decision-Making Approach

According to the HAM, the comprehensive decision-making approach is proposed based on combination weight and TOPSIS improved by Kullback-Leibler divergence. When the performance evaluation is achieved, it can provide support for the optimization selection of mRIFS fusion algorithms.

The determination of indicator weight is one of the key steps of comprehensive decision-making. Objective weighting and subjective weighting are two usual types of the determination methods. The latter is decision-makers’ subjective judgment of the importance degree of indicator’s influence on the target, which is the expression of decision-makers’ preference and is usually determined by AHP. Then the method of TOPSIS is usually used to sequence the objects in the weighted normalization decision matrix. As is shown in Figure 4, this paper has proposed a comprehensive decision-making approach of combination weighting and TOPSIS improved by Kullback-Leibler divergence based on the related research.

Figure 4 

The procedure of the comprehensive decision-making approach.

4.1. The Objective Weight

The objective weight is an abstract representation of the inherent relationship among all evaluation objects, which is determined by entropy method [26], standard deviation method [27], criteria importance through intercriteria correlation (CRITIC) [28], and so forth.

In this paper, there are indicators and evaluation objects, and every evaluation object represents the result of an experiment. So the indicator decision-making matrix is defined as , in which represents the value of evaluation object on the indicator   . To eliminate the effects of different dimensions between indicators, take the following standardized processing to obtain the standardized indicator decision-making matrix.(i)For the efficiency type indicator, .(ii)For the cost type indicator, .(iii)For the indicator which is better if it is more close to its mid-value, where

(1) Entropy Method. Entropy is a measure of the uncertainty of system state [29]. Assuming the system has different states and the appearance probability of each state is   , the entropy of the system is . When statuses have the same appearance probability of , the entropy is maximum; that is, max .

For the indicators, the entropy value of indicator is   according to the entropy theory. As gets bigger, the differences among the values of all evaluation objects on become smaller. It is generally believed that the indicator is more important if the differences among the values of all objects on are bigger. Using entropy method, the objective weight of the indicator is

(2) Standard Deviation Method. The principles of standard deviation method and entropy method are similar. If the standard deviation value of an indicator is bigger, the variation degree of the indicator is higher. So this indicator can play a more important role in the comprehensive decision-making, and its weight should be bigger. Using standard deviation method, the objective weight of the indicator is where is the standard deviation of the indicator .

(3) CRITIC. The idea of CRITIC is to determine the objective weight based on two concepts: the variability of an indicator and the conflict among indicators. The variability of the indicator represents the indicator value difference among all evaluation objects with the form of the standard deviation . If is bigger, the value difference of among all evaluation objects is greater. On the basis of the correlation between two indicators, the conflict among indicators can be expressed as , where is the correlation coefficient between the indicators and and are the arithmetic average value of and . If there is a strong positive correlation between two indicators, the conflict of them is low. The information volume of is determined through synthesis consideration of the variability of and the conflict between and other indicators. If an indicator has a larger information volume, it is more important. Using CRITIC, the objective weight of the indicator is where is the information volume of .

4.2. The Subjective Weight

Traditionally, the exact number is generally used in decision-makers’ judgments to express the relative importance between two elements of the AHP judgment matrix. Human beings are often with ambiguity, and using the “little,” “clear,” “strong,” and other vague concepts to express their purpose is more rational. Therefore, based on the integration of the trapezoidal fuzzy number and scale method, the indicators’ subjective weights are determined through the trapezoid fuzzy scale AHP.

The natural number can be converted to trapezoidal fuzzy number in turn based on the membership function of trapezoid fuzzy number, so that it is

When evaluation matrix is built by decision-makers’ fuzzy comment, the -level comparative scale method is often used to determine the element value of evaluation matrix, and is usually odd. To some extent, this method can reflect ambiguity of decision-makers but still cannot more “aptly” reflect the actual thinking. Therefore, with a division of two trapezoidal fuzzy numbers to improve the n-level comparative scale according to formula (7), we have the trapezoidal fuzzy scale value as formula (8). The comments of the 9-level comparative scale are “extremely inferior,” “strongly inferior,” “significantly inferior,” “slightly inferior,” “equal,” “slightly superior,” “significantly superior,” “strongly superior,” and “extremely superior”, and their trapezoidal fuzzy scale values are . Therefore,

The subjective weight is determined as follows.

Step 1. According to the definitions of the HAM shown in Figure 3, let , = LNIC, = RR, = FD, and survey    decision-makers of their views for . The importance degree of relative to given by the decision-maker is , , , so the fuzzy reciprocal judgment matrix of the decision-maker is .

Step 2. Do a consistency check on individually. The simple form of is , , in which represents the kern of . Assuming that , , let , so . If ( is usually equal to 0.1), passes the consistent check, so is a consistent matrix, or some appropriate adjustment of is needed.

Step 3. Assuming that decision-makers are equally important, construct the fuzzy reciprocal judgment group-decision matrix , , in which , , , and . Therefore, the fuzzy evaluation value of relative weight of is , where , so the membership function of is
Then the expectation of is , in which is the optimism factor and and are the inverse functions of on and , respectively. When , . Make the same calculation for other elements of ; then do normalization processing, so the relative weight of is .

Step 4. Assuming that TRPEM shown in Figure 3 has levels, repeat the above steps top-down.

Step 5. Assume that the relative weight vector of elements in level for the total target is and the relative weight vector of elements in level to the element in level is . In , the relative weight of elements not subject to element is zero. Let , so the relative weight vector of the elements in level to the total target is , in which is the relative weight vector of the elements in level 2 to the total target. Finally the relative subjective weight vector of the elements in level to the total target is calculated out, which is represented by .

4.3. The Combination Weight

According to the objective weight vector and subjective weight vector, the experience factor is introduced in the method, so the combination weight is

Here, the experience factor represents the decision-makers’ preferences for the objective and subjective weights. If decision-makers think that the objective weight is more important than the subjective weight, select ; otherwise select . If decision-makers think that both of them are equally important, select . The weighted normalized decision-making matrix is obtained, , .

4.4. Improved TOPSIS by Kullback-Leibler Divergence

In a limited number of evaluation objects, make a set of optimal indicator values and a set of the worst indicator values as the positive ideal point and negative ideal point, respectively, and sort evaluation objects with their proximity to the ideal point. This is the basic idea of TOPSIS. In traditional TOPSIS, Euclidean distance between the evaluation object and the ideal point is used to indicate the proximity, which cannot indicate the proximity of evaluation objects on the vertical line of the two ideal points and fully reflect the merits of all objects. Corresponding solutions mainly include improving the closeness by the cosine, replacing Euclidean distance with the vertical plane distance or vector projection, and partial sequence alignment, but these methods have some deficiencies.

Kullback-Leibler divergence is an asymmetric measurement of the difference between two probability distributions in information theory [30, 31]. The difference degree of two -dimensional uncertain systems and can be measured with KLD; that is, , , and . It can be proved that and when and only when .

Definition 8. The Kullback-Leibler divergence from evaluation object to the positive ideal point is , and the Kullback-Leibler divergence from evaluation object to the negative ideal point is , where    is the row of and , .

Definition 9. The Kullback-Leibler divergence closeness between and the ideal points is . We get the following.(i)If , .(ii)If , .(iii)If and , when ; that is, .

Therefore, replacing Euclidean distance with Kullback-Leibler divergence between evaluation object and the ideal points is reasonable, which coincides with the basic sort principles of TOPSIS, and can solve the problem of the above improved methods. So calculate the Kullback-Leibler divergence closeness between each evaluation object and the ideal points successively and get the final evaluation results by sorting them in the descending order [16, 17].

5. Case Study

Experiment conditions were as follows:(i)flight target type: airplane XX-50;(ii)radar type: radar XX-713;(iii)detection resolution: distance of 500 m, direction of 0.55°, and height of 0.83 m;(iv)random error: distance of 500 m, direction of 0.3°, and height of 0.

The first radar’s longitude is set to 120°0′26′′ and the latitude 25°0′32′′. The second radar’s longitude is set to 119°0′26′′ and the latitude 24°0′32′′. In the experiment, the number of flight targets is from 3 to 13, and the tracks include horizontal-flight and cross flight. As the data source, both the data collected by the two radars and the real flight data are put into the information integration testing laboratory platform.

The key of multitarget track is data association. For the association of trace points generated by radar data, the square gate and elliptical gate are used as the tracking gate, and the nearest neighbor data association (NNDA) and probabilistic data association (PDA) are used as the association algorithm.

After being associated with trace points, the tracking information is put into the track fusion module for track association and fusion, and the fusion result is also the object to be analyzed in this paper. KNN is used for track association, and SF is used for track fusion.

The indicators can be divided into three types as follows.(i)The efficiency type indicators include CRR (in theory, its minimum value is 0 and maximum value is 1) and CSR (in theory, its minimum value is 0 and maximum value is 1).(ii)The cost type indicators include nCTR (in theory, its minimum value is 0 and maximum value is the number of targets), SPEE (in theory, its minimum value is 0 and maximum value is 2 km), SSEE (in theory, its minimum value is 0 and maximum value is 40 m/s), ERR (in theory, its minimum value is 0 and maximum value is 1), and LRR (in theory, its minimum value is 0 and maximum value is 1).(iii)The indicator which is better if it is more close to its mid-value includes TRF (in theory, its minimum value is 0.7 and maximum value is 1.3).

The original experimental data is shown in Table 1.

After standardized processing, the standardized indicator decision-making matrix can be obtained as

5.1. The Objective Weight

Using entropy method, standard deviation method, and CRITIC, respectively, we can obtain three objective weight vectors shown in Table 2. Through the comparison of them in Figure 5, it is found that the assignment of objective weight determined by entropy method tends towards equilibrium, and the assignments of objective weight determined by standard deviation method and CRITIC are in general close. Entropy method, which reduces the competition of different indicators and makes the assignment of objective weight balanced, cannot fully reflect the objective weight assignment. Standard deviation method and CRITIC can reflect more competition information than entropy method. Taking both the variability of an indicator and the conflict between it and other indicators into consideration, CRITIC is more reasonable than standard deviation method. The analysis result is consistent with the above definition of these methods in Section 5.1. So we select CRITIC to determine the objective weight. The objective weight vector is .

Figure 5 

The subjective weights of all nodes relative to the upper node.

5.2. The Subjective Weight

Due to space limitations, we take the substandard level for an example using trapezoidal fuzzy scale AHP. LNIC, RR, and FD belong to the standard level node TAQ.

Step 1. Assuming that there are three decision-makers, the fuzzy reciprocal judgment matrices are

Step 2. After calculation, all of them can pass the consistency check and can be used to calculate the subjective weights.

Step 3. The three decision-makers are equally important; the fuzzy reciprocal judgment group-decision matrix is
The fuzzy evaluation values of relative weight of LNIC, RR, and FD are 0.2713, 0.3700, 0.6822, and 0.9655, 0.1292, 0.1803, 0.3230, and 0.4262, and 0.1491, 0.1980, 0.3310, and 0.4277 successively. Selecting , the expectations are 0.5723, 0.2647, and 0.2764. After normalization processing, the subjective weights of LNIC, RR, and FD relative to TAQ are 0.5140, 0.2377, and 0.2483.