Abstract
Circular Error Probability (CEP) is defined as the radius of a circle where the probability of an impact point being inside is 50%, which is also widely used as a measure of the guidance weapon systems’ precision. In order to achieve a fusion of various test information, Bayesian methods and improved Bayesian methods have been extensively studied in calculating the CEP. Nevertheless, these methods could fail when there exists unknown systematic bias in the prior information. Therefore, a novel method called Bayesian estimation based on representative points (BERP) with an optimization procedure for determining the optimal number of representative points is proposed in this paper. Explicit theoretical analyses demonstrate that the BERP outperforms the classical Bayesian methods when fusing the slightly biased prior information and also give the bound of the systematic bias for stopping using the heavily biased prior information. Moreover, when the systematic bias is within the bound, simulation results indicate that our method is credible and outperforms the classical Bayesian method in calculating the CEP of guidance weapon systems.
1. Introduction
Performance evaluation of complex equipment, such as the guidance weapon systems, is very important before application. In the assessment of impact accuracy of the guidance weapon systems, CEP is the most commonly used as a measure, which can integrate the precision and dispersion to assess the impact accuracy [1–3]. Usually, the tighter the pattern of impact point errors, the smaller the CEP we can get; i.e., there is higher impact accuracy of guidance weapon systems. Let be the impact point errors of the projectile, where and are downrange and cross-range misses, respectively; then the CEP can be defined as follows [4, 5]:It is shown in the above equation that the CEP can be calculated by numerical integrations when giving the probability density function . In practice, are usually assumed to follow the bivariate normal distribution, i.e., ,where and are the mean and the covariance, respectively. Then the problem of calculating CEP rests on estimating and , whose accuracy will determine the precision of the CEP calculation.
In the performance evaluation of guidance weapon systems, the impact points used to calculate the CEP are collected from various tests. The realistic tests of guidance weapon systems are usually extremely expensive and time-consuming, so generally the sample size of impact points is very small. Moreover, the calculation of CEP based on the small sized data would lead to unreliable results, so it is reasonable to introduce the prior information to refine the results. Therefore, Bayesian estimation is widely used to achieve the fusion of the prior information and the realistic test information [6, 7] and hence to increase the reliability and the accuracy of the estimation of , as well as the CEP. In fact, the most important prior information is provided by the substitute tests in the development processes. The performance evaluation is a sequential process, so the data collected from substitute tests are usually regarded as prior samples. In order to use the prior information more reasonably, several strategies are introduced in bringing the credibility of the prior information into Bayesian estimation, such as data compatibility tests, the information divergence, and the theory of fuzzy operators [8, 9]. In the CEP calculation, Huang comes up with a measure of credibility from physics resources of data [10], which has a good estimation accuracy if we have a clear understanding of the physical background. Duan suggested a method with all prior information normalized into one test sample [11], which can reduce the deviation when prior information got distorted, but more theoretical discussion is needed for this method when considering the information loss and fusion efficiency of the normalization.
However, the substitute tests for the guidance weapon systems may be systematically biased compared with realistic tests. Since the pattern of systematic bias is unknown, we cannot estimate the bias but can only give a rough range of it. Simulation results demonstrate that the systematically biased prior information will cause serious impact on the mean estimation when applying classical Bayesian estimation directly. The estimation of the parameter is slightly affected by the unknown systematic bias because of the same guidance system of these tests. The improved methods considering the normalization of prior information could reduce the estimate bias of , but this will cause much loss of prior information and inevitably leads to an unreliable calculation of CEP. One possible way to solve the problem is by choosing appropriate samples, such as representative points to generate new prior information.
Representative points (RPs), also known as Principal Points [12], are a group of points that could represent a distribution with the least mean squared error (MSE). RPs’ theory is brought up in 1990s and now widely applied in clustering analyses [13], statistical simulations [14], image processing [15], and so on. In this paper, we resample samples from the systematically biased data to get the RPs which are regarded as the new prior information, expecting to reduce the estimate bias of substantially. This new estimation method is called Bayesian estimation based on representative points (BERP). Meanwhile, we propose an optimization procedure which balances the effects of estimate bias and information loss to determine the optimal number of RPs. In addition, two theorems are proposed to prove that the estimate bias and MSE of with RPs are smaller than those with the raw data. Furthermore, we also analyze the bound of the systematic bias for stopping using the heavily biased prior information. Within the bound, both the simulations and authentic experiments show that our BERP outperforms the classical Bayesian methods in estimating the parameters. In the performance evaluation of guidance weapon systems, via BERP it is better to choose RPs from the raw prior samples as new prior information when calculating the CEP. On the whole, by using the theory of RPs, our works enrich the Bayesian methods especially on the prior information fusion patterns theoretically and provide a possible way to solve the engineering problem in performance evaluation of guidance weapon systems.
The rest of this paper is organized as follows. Section 2 introduces the classical Bayesian estimation applied in CEP calculation. Section 3 describes some notations and preliminaries of RPs, the process of BERP, and the optimization procedure for determining the optimal number of RPs. In Section 4, we propose two theorems to compare the estimation performances by BERP and classical Bayesian estimation and analyze the bound of systematic bias for stopping using the biased prior information. Section 5 shows three numerical experiments about our new method. The conclusion is given in Section 6.
2. Bayesian Estimation in CEP Calculation
2.1. Calculation of CEP
Suppose that follows the bivariate normal distribution , whereThe parameters , are standard deviations of impact point errors for downrange direction and cross-range direction, respectively; , are means of impact point errors for each direction; () is the correlation coefficient of and . In most cases, and are independent of each other. Nevertheless, even if , we can use the orthogonal transformation to achieve the decorrelation of . So we assume that in the rest of the paper. Under the assumption that , the CEP satisfies the equation
After estimating the parameters , , , and , the CEP can be calculated by numerical integrations. More details about calculating CEP are given in [16].
2.2. Classical Bayesian Estimation
As described in the Introduction, the performance evaluation of guidance weapon systems is a sequential process; the distribution parameters would change when fusing test data from different stage. The realistic tests are conducted to refine the previous evaluation results based on the substitute tests. Therefore, the data collected from substitute tests are usually regarded as prior samples.
Because the procedures of estimating the parameters and are similar, we take the downrange direction of impact point errors as an example to introduce the classical Bayesian estimation, where follows the normal distribution . For convenience, we drop the subscripts of as and let be the joint prior distribution of . In Bayesian theory, the conjugate prior distributions of and are normal distribution and inverse Gamma distribution , respectively. As for the distribution parameters , , , and , they are determined by (7). The probability density function of iswhere means the Gamma function. Moreover, the joint prior distribution is the normal-inverse Gamma distribution, so we haveSuppose the prior samples for downrange direction are , is the size of the samples; letThen the estimates of the parameters of the joint prior distribution areSimilarly, when the realistic test samples are obtained, is the size of the samples; letBecause of the property of conjugate prior distribution, the posterior distribution of is also a normal-inverse Gamma distribution:where , , , and are the parameters of the normal-inverse Gamma distribution, and the estimates of the parameters areSo the estimates of by classical Bayesian estimation are
As for the cross-range direction of impact point errors, the estimates of and can also be calculated by (11). After obtaining the estimates of , , , and , we can calculate the CEP of impact points by (3).
3. Bayesian Estimation Based on Representative Points
In this section, we propose the novel method Bayesian estimation based on representative points and the procedure of determining the optimal number of RPs. RPs can not only optimally represent the distribution of prior information in terms of MSE principle, but also have smaller sample size compared with raw prior samples. Therefore, RPs can retain the useful information of prior samples and reduce the estimate bias of , . If we search RPs as new prior information, we may get more accurate and reliable CEP of guidance weapon systems.
3.1. Methods for Searching Representative Points
In this subsection, we will give a brief introduction of RPs and methods for searching RPs. Assume that is a dimensional random vector, and the probability density function of is . Define the mean squared error for a set of points of the random vector as follows:where stands for -norm. The vectors are called representative points of a random vector iffor all sets .
From the definition above, it is obvious that when , the single RP is the mean of . Searching the RPs equals doing the optimal grouping [17]; it is difficult to derive the concrete RPs theoretically even if the number of RPs is given. Flury has proved that there is no theoretical derivation of RPs when [18]. So some approximation algorithms have been proposed to search RPs including k-means methods [19], parametric k-means methods [20], and nonparametric methods [21]. The k-means methods are searching the clustering centers as the RPs. Moreover, the parametric k-means methods resample large samples from a specific distribution whose parameters are estimated by maximum likelihood, then searching the RPs from the resampled samples by k-means methods. The main idea of nonparametric methods is to build the empirical distribution function of and resample large samples from this empirical function, after which the RPs are chosen from the resampled samples. In most cases, nonparametric methods have better performance to represent a distribution in terms of MSE than the k-means methods and parametric k-means methods. The main steps of the nonparametric method introduced in [21] are shown as follows:(i)Step1. For original samples , use k-means algorithm to obtain points from as an initial solution.(ii)Step2. Use the kernel estimation method to estimate the density function of original samples, denoted as .(iii)Step3. Use the Randomized Likelihood Sampling method to generate samples from the density function as the training data.(iv)Step4. Based on the training data and the starting points , use k-means algorithm to obtain RPs .
Following the four steps above, we can get the RPs from prior samples. After that, we can use the RPs as new prior information to estimate the population parameters. More details about BERP will be introduced in the next subsection.
3.2. The Procedure of BERP
In this subsection, we will introduce the procedure of BERP which is similar to the classical Bayesian estimation in Section 2.2. We also take the downrange direction of impact point errors as an example, suppose the original prior samples are . Choose RPs from by the nonparametric method introduced in Section 3.1 and denote , and letSimilar to Section 2.2, let , , , and be the parameters of the joint prior distribution. By searching RPs as new prior information, the estimates of these parameters areMoreover, let , , , and be the parameters of posterior distribution. So the estimates of these parameters arewhere and are calculated by (8). So the estimates of by BERP are
If we know the exact number of RPs, it is easy to estimate the parameters , , , and by (17). However, it is difficult to determine the optimal number of RPs because there are no theoretical methods about this. Therefore, we propose an optimization procedure to determine the optimal number of RPs when there exists unknown systematic bias in prior samples.
3.3. Optimal Number of Representative Points
The approach to determining the optimal number of RPs varies with the background of practical problem. On the one hand, the RPs are closer to the original prior samples as the number grows. Therefore, the more the RPs chosen, the larger bias they may bring to the estimate of , which will also be validated in Theorem 1, Section 4. On the other hand, similar to the methods of normalizing prior information, choosing small number of RPs will cause great information loss. Therefore, we consider two factors when determining the optimal number of RPs: the estimate bias and the information loss. We still take the downrange direction of the impact point errors as an example to describe the optimization procedure. and stand for the estimate bias and the information loss, respectively, where stands for the number of RPs. The objective function is to achieve a balance between and . So we have
In fact, it is hard to quantify the estimate bias without the true values of the population parameters. But there is a principle in the performance evaluation of guidance weapon systems that every realistic test sample should be used. So we can use the mean of realistic test samples to approximate the true value of parameter , denoted as . The estimate of by BERP is denoted as . So the approximated isAs for , the information loss could be estimated by Cox (1957) [17]:where is the variance of all prior samples; is the variance of samples in class which is classified in for searching RPs. As the number of RPs increases, the estimate bias would increase and the information loss would decrease. The optimal number of RPs is determined by the minimal value of the objective functionAlgorithm 1 describes the optimization procedure of determining the optimal number of RPs.
| ||||||||||||||||
4. Theoretical Analysis of BERP
Estimate bias and MSE are the common measures to evaluate the quality of an estimator. So we will use them to analyse the theoretical performance of BERP and classical Bayesian estimation in this section. Suppose the posterior distribution of parameter is , is the posterior expectation of , and is the true value of . The estimate bias and MSE of the estimator arewhere is the variance of .
Let realistic test samples follow the normal distribution , and the prior samples follow the normal distribution , where is the systematic bias. Let , be the sample sizes of prior samples and realistic test samples, respectively. Suppose the posterior estimate of by classical Bayesian estimation is , is its posterior expectation. The posterior estimate of by BERP is , is its posterior expectation, and is the number of RPs. Let be the true value of parameter . There are two theorems to compare the estimate bias and MSE of the two estimators and .
Theorem 1. In the case that there exists systematic bias in prior samples, one has
Proof. From (11) and (17), the estimate bias of and is The estimate bias of increases with the number of RPs. Since the size of prior samples is much larger than the size of RPs, we have . Therefore there is So
Theorem 2. In the case that , one has
Proof. The MSE of and arewhere Because prior samples and realistic test samples both follow the normal distribution and there exists systematic bias in prior samples, we have the approximated results: Let and , and is determined as follows:when , if and only if . Because , , we can get . To sum up, when , there is
The estimate bias of is smaller than that of when there exists unknown systematic bias in prior samples. Moreover, the MSE of is also smaller than that of when . In most cases, is smaller than . Therefore, it can be concluded that BERP has better accuracy for estimating the parameter of normal distribution than classical Bayesian estimation when there exists unknown systematic bias in prior samples.
However, when the systematic bias is beyond a certain bound, it may be better to stop fusing the biased prior samples even if they may provide some useful information. Suppose is estimated by maximum likelihood estimation without using the prior information, it is obviously an unbiased estimate. The MSE of is
when , if and only if , where
Moreover, compared with (34), there is , which means that when , is also smaller than . Therefore, if is larger than , we should stop using the prior information; is the bound of the systematic bias.
5. Numerical Experiments
In this section, three numerical experiments are provided to validate that BERP can help to get more accurate and reliable calculation of CEP when the systematic bias is within the bound. The first one is to show the performance of optimization procedure when determining the optimal number of RPs; the second one is to compare the estimation accuracy of BERP with that of classical Bayesian estimation; the third one is to compare the calculation of CEP based on BERP with classical Bayesian estimation.
Example 1. In this example, we will analyze the optimization procedure for determining the optimal number of RPs. Let be the sample size of the prior samples and be the sample size of realistic test samples. The prior samples and the realistic test samples are generated from normal distributions and , respectively, where . The RPs are searched by the nonparametric method. In order to reduce the influence of random factors on simulation result, resample samples 100 times under the same circumstance and get the average results. Figure 1 shows the variation of the information loss, the estimate bias, and the objective function for different numbers of RPs, where the estimate bias and information loss are both calculated after normalization.
As displayed in Figure 1, the estimate bias approximated by (19) is very close to the theoretical one, which means that using to quantify the estimate bias is reasonable. In addition, when the number of RPs increases, the information loss decreases while the estimate bias increases, so the objective function can balance the two factors well when determining the optimal number of RPs. From Figure 1(b), when the number of RPs is 9, the objective function reaches the minimal value, so the optimal number of RPs is 9 in this example.
(a)
(b)
Example 2. In this example, we will compare the estimation performance of the two methods. Let be the sample size of the prior samples, be the sample size of the realistic test samples, and be the predetermined maximum number of RPs. The prior samples and realistic test samples are generated from normal distributions and , respectively, where and is the systematic bias in prior samples. We set different values for in the simulations to investigate the estimation performance when estimating the parameter . Similar to Example 1, we resample samples 100 times under the same circumstance and get the average results in order to reduce the influence of random factors. Table 1 shows the simulation results.
From Table 1, we can summarize two conclusions. Within the bound of the systematic bias shown in Section 4, if there exists slight systematic bias in prior samples, is much more closer to the true value of the parameter than , and is also smaller than in most cases. Therefore, BERP has higher accuracy to estimate the parameter than classical Bayesian estimation when there exists slight systematic bias in prior information. If there is no systematic bias in prior samples, is very close to , and is a little larger than . There is no obvious difference between the estimation accuracies of the two methods. Moreover, without the systematic bias in prior samples, the optimal number of RPs is close to the predetermined maximum number 20. In this case, BERP is degenerated into the classical Bayesian estimation to some extent.
Example 3. When there is systematic bias in prior information, the CEP calculation based on BERP and classical Bayesian estimation is discussed in this example. We simulate prior samples from the bivariate normal distribution and realistic test samples from the bivariate normal distribution , where , and . The parameters , , , and are estimated by BERP and classical Bayesian estimation, respectively. Based on the estimates of the population parameters, we use the numerical integration to get the CEP. Figure 2 shows the simulation results about CEP calculation.
As shown in Figure 2, the true CEP is , the CEP calculation based on BERP is , and the CEP calculation based on classical Bayesian estimation is . It is easy to conclude that the CEP calculation based on BERP is much closer to the true CEP when the systematic bias is within the bound. In addition, the CEP calculation without using the prior information is defined as . If the prior information were effectively ignored, the would be unreliable and unstable because of the large MSE of the estimation of without using prior information. Therefore, using BERP to estimate the parameters , will get more accurate calculation of CEP than classical Bayesian estimation when there is slight systematic bias in prior information. In addition, CEP calculation by BERP also outperforms the method with the prior information ignored.
6. Conclusions
In this paper, we have investigated the methods for performance evaluation of guidance weapon systems. Because of the small sample size of the realistic test data, we would fuse the prior information in the evaluation. However, by classical Bayesian estimation, the unknown systematic bias in prior information may cause large deviation for CEP calculation. For purpose of addressing it, a novel Bayesian method called BERP is proposed in this paper, and the corresponding optimization procedure is designed. In addition, we also give the bound of systematic bias for stopping using the heavily biased prior information.
Within the bound of the systematic bias, theoretical analysis and simulation results prove that our new method has smaller estimate bias and MSE for estimating the mean of normal distribution than classical Bayesian estimation when there exists slight systematic bias in prior information. As for CEP calculation, the simulation results also validate that the CEP calculated by BERP is more accurate and reliable than the CEP calculated by classical Bayesian estimation. It can be concluded that a more accurate and reliable estimation of the CEP can be obtained via the BERP when the unknown systematic bias is within the bound.
There is no obvious difference of the estimation accuracy between the two methods; BERP also has a good estimation performance when there is no systematic bias. Therefore, in order to get accurate and reliable evaluation results of guidance weapon systems, it is better to calculate the CEP via BERP than classical Bayesian estimation if the systematic bias is within the bound. In contrast, if the systematic bias is beyond the bound, we should stop fusing the biased prior information and evaluate the performance only by realistic test samples even if the sample size is small.
Data Availability
The simulation data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
All authors have contributed to the study and preparation of the article. Bowen Liu and Xiaojun Duan conceived the idea, derived equations, and did analysis. Bowen Liu and Liang Yan finished the programming work and wrote the paper. All authors have read and approved the final manuscript.
Acknowledgments
This work is supported by National Natural Science Foundation of China (no. 11771450).