Abstract
The evaluation for autonomous capability of ground-attack unmanned aerial vehicle (UAV) comes from the demand of reality, which determines the operational use of airborne equipment authority. It essentially entails a multicriteria decision-making process accounting for evaluation model and uncertainties. Firstly, as for the construction of evaluation model, the index model is proposed from four aspects of observation capability, decision capability, action capability, and security capability, namely, ODAS, which analogizes cognitive behavior mechanism of human based on airborne equipment; then, to solve uncertainties of randomness and fuzziness in the process of autonomous capability evaluation, a cloud model approach is proposed, which expresses uncertainties by the certainty degree distribution. Finally, the cloud model-based approach is tested by evaluating typical UAVs and comparing with Hopfield neural network method. The results show that the evaluation of the autonomous capability based on the cloud model is accurate and more representative than the Hopfield neural network method.
1. Introduction
The ground-attack UAV is ranked as one of the main combat equipment in the future. The complex operational tasks and variable operational environment make it necessary for UAV to have a high degree of autonomous capability [1]. In the actual battle, the completion of tasks requires the effective man-machine division between ground station and UAV system, which depends on the classification of the autonomous capability level. Moreover, with the rapid development of artificial intelligence theory, it has changed the operating rules of the previous unmanned system and made the autonomous system more smart and efficient. For example, machine learning algorithms can analyze and predict sudden communication problems in uncertain and complex scenes. Applying machine learning algorithms to communication systems can make systems more intelligent and autonomous [2, 3]. Therefore, it is of great military significance to scientifically establish an evaluation system for the autonomous capability of ground-attack UAV and formulate the criteria for grading autonomous systems with the national conditions, which can improve the combat effectiveness of equipment systematically.
At present, the research on evaluation model of autonomous capability is in its infancy both here and abroad. A general survey of evaluation models shows that typical ones include Sheridan’s levels of automation (LOA) [4], autonomous control level (ACL) [5], autonomy levels for unmanned systems (ALFUS) [6], four-level model in man-machine division [7], and reference frame of autonomous system [8]. These models apply to particular conditions and have certain limitations. For instance, with the vague classified implication, ACL divides the UAV system into 10 grade standard and only reflects the relative autonomy degree between different UAVs. Although some progress has been made in the evaluation model abroad, there is still a big gap in the quantitative evaluation. It does not analyze autonomous capability for a particular equipment, nor does it elaborate on the source of indicators. Therefore, the ODAS model is proposed for ground-attack UAV by analoging man-machine cognitive model. With previous research [9], it explains the source of evaluation indicator in detail and is ready for quantitative evaluation of ground-attack UAV.
Due to the uncertainty of physical cognition and monitoring information of UAV system, the key to quantitative evaluation of autonomous capability is to deal with deviation caused by uncertainty. There are two types of uncertainty that should be considered in autonomous capability evaluation: (1) randomness, which is often exhibited in the monitoring and analysis of data-related autonomous capability; and (2) fuzziness, which is often reflected in the evaluation of classification standard, evaluation class. Traditional evaluation methods usually only consider the fuzziness or randomness and ignore another in evaluation information. For instance, the fuzzy AHP was used to study the autonomous evaluation level of unmanned systems in literature [10] and q-Gaussian fuzzy neural network was used to evaluate the operational effectiveness of aircraft in literature [11], which ignore the importance of randomness; Instead, particle swarm optimization method was used to evaluate water quality in literature [12] and simulated annealing optimization algorithm was used to access land ecological security in literature [13], which ignore the importance of fuzzy.
A new hybrid model considering randomness with fuzziness, namely, cloud model [14], is an efficient cognitive technique which is expressed with three fixed parameters-mathematical expectations, entropy, and superentropy. Based on the normal distribution and a certainty function, the normal cloud mode quantifies both randomness and fuzziness by means of three fixed parameters, being more advantageous than single randomness or fuzziness type models. The normal cloud model has been widely applied in information science, such as water quality assessment [15], risk assessment of water inrush in karst tunnels [16], sustainability assessment on mineral resources [17], and target threat level assessment [18]. Above evaluations, making use of the cloud model is efficient to solve relatively random and fuzzy issues.
Based on the abovementioned evaluation model and evaluation method review, the primary motivations of this study are outlined as follows:(1)Through extending the man-machine cognitive behavior, a new model, namely, ODAS model, is proposed, reflecting the central concept about autonomy, which has no official definition. Moreover, evaluation index is selected from actual airborne equipment, which has more practical significance to develop key equipment technique.(2)It is the first time to achieve quantitative evaluation of autonomous capability by introducing cloud model theory in this field. Furthermore, in order to prove the validity and rationality of evaluation theory, the simulation is tested by typical UAVs in this paper.(3)In order to reflect subjective differences of experts and objective differences of indicators, a new weight method regarding improved distance measure approach combined with entropy weight is proposed. The similarity is calculated by the distance of cloud drop rather than the traditional method based on ordinate of cloud drop.
To sum up, the rest of the paper is organized as follows: Section 2 establishes evaluation index system integrated with the ODAS model on the basis of analyzing the performance parameters of airborne equipment and introduces basis theory of cloud model theory. Section 3 discusses the key issues in evaluation process and provides appropriate solutions. Section 4 illustrates the application of the technique to typical UAV in America and assesses the validity of the proposed cloud model-based approach by comparison with other methods. The conclusions are given in Section 5.
2. Model and Methodology
2.1. Evaluation Model
Autonomous capability of UAV refers to the ability of unmanned systems to perceive, observe, analyze, communicate, plan, make decision, and take action [1], which means that it can sense the external situation online, make decision in flight as scheduled, and carry out the mission autonomously. The development of autonomous capability of UAV system is a gradual process. From remote control flight to autonomous flight and man-in-loop to man-out-loop, it reflects the change of man-machine relationship. The US Air Force Research Laboratory shows that the highest level of autonomous capability is fully autonomous swarms, which does not mean that the UAV can be completely out of human control, and the human still has the authority to monitor the status of UAV. From the perspective of man-machine joint cognition, the man-machine relationship is shown in Figure 1. UAV system depends on human beings to carry out navigation, guidance, control, and other tasks. Only using mechanical and electrical equipment to replace human function can the fully autonomous operation be realized.
Therefore, the core of autonomous system is to replace human function in uncertain environment. The only way to obtain this capability is that UAV has a human-like behavior mechanism [19]. The human nervous system is a perfect autonomous system, which has three layers of hierarchical structure to support three different types of behavior mechanisms: declarative behavior, procedural behavior, and reflective behavior. Declarative behavior refers to the reasoning, judgment, and other high-level cognitive behavior, which is the most important executive body of decision-making. Procedural behavior refers to the proficiency behavior produced by the middle nerve center, which is responsible for the transmission of instructions and sensing information between the upper layer and the lower layer. Reflex behavior is the control and execution of concrete behavior by the underlying nerve center, which is the most important generator of actual control behavior.
Mapping declarative behavior, procedural behavior, and reflective behavior to structure layer of autonomous system, it extends four kinds of capabilities, namely, ODAS, from observation capability, decision capability, action capability, and security capability in Figure 2. Among them, observation capability and security capability respond to procedural behavior; decision capability responds to the declarative behavior and action capability responds to reflective behavior. Based on the above capabilities, an evaluation model of ODAS is proposed hierarchically.
Based on the ODAS model and previous research on the source of evaluation indicator in detail [9], 31 equipment parameters are selected as evaluation index for autonomous capability of ground-attack UAV. Evaluation index system is established according to the performance parameters of existing airborne equipment, which can distinguish the differences of key technologies. Dividing evaluation index system into three layers, goal layer represents four kinds of capabilities extended by the man-machine cognitive model; subsystem layer represents relative airborne equipment extended by the goal layer, and index layer represents relative performance parameters extended by the subsystem layer. Evaluation index system is shown in Table 1.
2.2. Evaluation Methods
2.2.1. Definition
Cloud model theory is a modern mathematical theory for the study of compound uncertainties, which realizes the transformation between qualitative concepts and quantitative data based on probability statistics and fuzzy set. Let be the universe of discourse and be a qualitative concept in . If is a random numerical realization of , which satisfies ,, the certainty degree of belonging to concept satisfies
Then, is called cloud drop and the distribution of in the universe is a normal cloud [20].
In this way, the normal cloud model can effectively integrate the randomness and fuzziness of concepts by the three parameters of expectation , entropy , and hyperentropy . As shown in Figure 3, taking the cloud model ,, and as example, represents the highest point in the cloud model, which can represent qualitative concepts in the universe; is the uncertainty measure of , reflecting the acceptable range of cloud drop in the universe; is the uncertainty measure of , reflecting the dispersion degree of cloud drop [21].
2.2.2. Forward Cloud Generator
Forward cloud generator is a tool that converts qualitative concepts into quantitative values and generates cloud drop in accordance with a probability distribution through parameters . Forward normal cloud model with N drops can be generated by the following steps:(1)Generate a normally distributed random number with expectation and variance .(2)Generate a normally distributed random number with expectation and variance .(3)Calculate ; the coordinate of the cloud drop is .(4)Finally, repeat step 1–step 3 until N cloud drops are generated.
As shown in Figure 4, considering the importance of different cloud models in the universe, multiple cloud models are combined into a comprehensive cloud model, namely, weighted cloud model. Let be the weighted cloud model and , … be the N cloud models in the domain. The calculation formula is as follows [22]:where is the weight set of cloud model in the universe. When , the weighted cloud model is the average cloud model. For this article, we use the average cloud model to combine multiple cloud models into a comprehensive cloud model. Therefore, the weight of indicator cloud model is same. We calculate the weight of cloud model from the number of indicator cloud models. The formula is as follows:where is the number of indicator cloud models.
3. A Cloud Model-Based Assessment Approach
According to the established evaluation index system in Section 2.1, the cloud model-based assessment approach can be illustrated in Figure 5. However, three outstanding issues exist in the process when employing normal cloud model to quantitative evaluation of autonomous capability:(1) how to define the ideal bilateral boundaries on given levels (I, II, III, IV, and V) of autonomous capability; (2) how to calculate the weight of evaluation index; and (3) how to express the certainty degree of evaluation index.
3.1. Bilateral Boundary
The division of bilateral boundary concerning indicator level is the key to confirm parameters of normal cloud model. Dividing evaluation indicator into five levels, grade I is the lowest level and grade V is the highest level. According to the existing database, 200 data of evaluation indicator are selected by bubble sorting and are divided into five intervals, taking the average value of five intervals as bilateral boundary on given levels (I, II, III, IV, and V). The bilateral boundaries are shown in Tables 2–6 with previous research [9]; it explains the source of evaluation indicator in detail. In order to make the parameters of different airborne equipment comparable, the data are processed dimensionless by means of logarithmic method, weighted arithmetic, and exponentiation. At the same time, the addition rule is used to express the relationship between each capability and underlying indicators.
3.2. Index Weight
Index weight reflects the contribution of evaluation index to autonomous capability, and the difference among index weight is mainly due to three aspects: (1) the experts attach different importance to evaluation index, reflecting subjective differences of experts, (2) the essential function of each index is different, reflecting the objective differences among the indicators, and (3) the reliability of evaluation indexes is different, reflecting the different reliability about information provided by each index. Based on the abovementioned problems, the index weight is calculated by a new method about the combination of distance measure approach and entropy weight method. Distance measure approach considers subjective factors on the index weight of subsystem layer mentioned in Section 2.1; entropy weight method considers objective factors and reliable degree on the index weight of index layer.
3.2.1. Distance Measure Approach
Let evaluation index be , experts be , and the weight of experts on index be . For qualitative indicator, it is ambiguous for experts to give specific weight value directly. Therefore, is expressed as by the forward cloud generator and distance measure approach is used to calculate , according to the similarity between cloud model and average cloud model . The formula of average cloud model is as follows:
In order to compare the similarity between cloud model and , the distance measure approach is proposed in the universe. In the cloud model and , cloud drops are selected in interval . According to the bubble sort of index value, the certainty degree of cloud drops is stored in space and , respectively. Compared to traditional methods [14], the distance between cloud model and relies on the distance of cloud drop rather than the ordinate of cloud droplet. The calculation formula of the distance measure between cloud model and is as follows:where is the certainty degree of cloud model ; is the certainty degree of cloud model . Therefore, the similarity between cloud model and average cloud model can be given as follows:where represents the range of valid universe. Then, the expert weight and average weight given by all experts can be calculated by
3.2.2. Entropy Weight Method
Here, a comprehensive weight calculating algorithm coupled with entropy is proposed, which is expected to balance the potential subjective factor of distance measure method. The notion of “entropy” taken from the theoretical foundation of modern information theory means the change degree of evaluation index. Standardizing evaluation index to and forming data matrix, , aswhere represents the standard value of ith indicator on jth sample set; is the number of indicators; is the number of sample sets. In autonomous capability assessment, entropy under ith indicator can be calculated bywhere represents the uncertainty of sample data with n potential statements; is the frequency of the jth statement. Now, the combined weight in ith indicator can be attained, based on average weight mentioned in Section 3.2.1 and entropy-based weight in ith indicator , as
3.3. Certainty Degree
Before calculating the certainty degree of evaluation index, cloud model of each indicator should be generated. According to the bilateral boundary provided in Tables 2–6, the cloud model parameters are obtained by the following formula (one has [23]):where is the level of autonomous capability; bilateral boundary of the form is the interval value corresponding to a certain autonomous capability level; is usually a constant which represents the degree of ambiguity, generally 0.001–0.1 [24]. As shown in Figure 6, the cloud model under given levels (I, II, III, IV, and V) is generated by the forward cloud generator, in which the horizontal coordinate represents the index value and the vertical coordinate represents the certainty degree.
Then, bringing indicator value into generated cloud model, the certainty degree set is calculated by Section 2.2.2 under given levels (I, II, III, IV, and V). In order to eliminate outliers, the distribution function of certainty degree is approximated by fitting curve. The maximum value of distribution function is taken as the maximum certainty degree of .Combining index weight with the maximum certainty degree , the comprehensive certainty degree is given ashere is taken as the autonomous capability level under comprehensive certainty degree .
4. Application: Case Studies
4.1. Metrics
Based on the ODAS model, this section explains the metrics of autonomous capability level of ground-attack UAV in detail. The autonomous capability of ground-attack UAV is divided into five levels, with the specific content of each level being mapped to autonomous capability by referring to literature [25].The metrics of autonomous capability are shown in Table 7.
4.2. Initial Results
The cloud model-based approach is used to evaluate autonomous capability levels of three representative UAVs. The weights of evaluation index are shown in Table 8.
After calculating index weights, the certainty degree distribution of each level under all indicators is calculated according to Section 3.3. Due to the large number of evaluation indicators, only certainty degree distributions on given levels (I, II, III, IV, and V) of weather threat observation capability are illustrated in Figure 7 as a reference.
(a)
(b)
(c)
From Figure 7, we obtain different distribution patterns of certainty degrees at each level of weather threat observation capability. Not all levels have distributions for most cases and the distribution ranges are also different for those levels which have distributions. Certainty degrees at given levels are determined by the mean of certainty degrees obtained from 2000 simulations. The final autonomous capability levels are calculated by maximum certainty degree indicating the most probable value.
4.3. Results by Reference Method
Based on previous research [26], the Hopfield neural network method is used to compare autonomous capability levels of typical UAVs with cloud model theory. Index coding and simulation result of test samples based on Hopfield neural network are shown in Figures 8 and 9. Meanwhile, the compared evaluation results are listed in Table 9.
The evaluation method based on cloud model theory provides the membership distribution diagram of evaluation index under index grades, which enables us to feel the difference between standard grade value and evaluation index value more intuitively. However, the Hopfield neural network method directly provides specific evaluation result, which cannot provide more difference in detail. The results show that autonomous capability levels of UAV1, UAV2, and UAV3 are grades III, I, and II. Compared with the Hopfield neural network method, the evaluation method based on cloud model theory is more representative than the reference method.
5. Conclusions
Based on cognitive control behavior, the ODAS model is proposed to select 31 evaluation indicators from airborne equipment affecting autonomous capability of ground-attack UAV. Combining with the weight of evaluation index, a cloud model-based approach is applied to autonomous capability evaluation. Confirming autonomous capability level of ground-attack UAV by certainty degree of cloud model avoids the influence of subjective factors, and cloud model-based approach also reveals the distinction of autonomy at each level, which provides much more information on autonomy status than Hopfield neural network methods. Hence, the autonomous capability level calculated by cloud model-based approach is found to be more actual and representative than Hopfield neural network methods. At present, the findings of this paper are being applied to autonomous evaluation software.
Data Availability
The evaluation index level partition data, the detailed information of generated cloud model data, and the distribution of certainty degrees in concept cloud data used to support the findings of this study are included within the article. The code data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
The authors would like to gratefully acknowledge the support provided by Aeronautical Science Foundation Project (20155896025) and Air Force Engineering University Principal Fund (XZJY2019049).
Supplementary Materials
The source of evaluation index in the Supplementary Material for the explanation of evaluation index in Tables S1, S2, S3, and S4. Table S1: indicators of observation capability. Table S2: indicators of decision capability. Table S3: indicators of action capability. Table S4: indicators of security capability. (Supplementary Materials)