Abstract
By considering the complex networks, the cooperative game based optimal consensus (CGOC) algorithm is proposed to solve the multi-UAV rendezvous problem in the mission area. Firstly, the mathematical description of the rendezvous problem is established, and the solving framework is provided based on the coordination variables and coordination function. It can decrease the transmission of the redundant information and reduce the influence of the limited network on the task. Secondly, the CGOC algorithm is presented for the UAVs in distributed cooperative manner, which can minimize the overall cost of the multi-UAV system. The CGOC control problem and the corresponding solving protocol are given by using the cooperative game theory and sensitivity parameter method. Then, the CGOC method of multi-UAV rendezvous problem is proposed, which focuses on the trajectory control of the platform rather than the path planning. Simulation results are given to demonstrate the effectiveness of the proposed CGOC method under complex network conditions and the benefit on the overall optimality and dynamic response.
1. Introduction
In order to execute the missions such as simultaneous strike, cooperative reconnaissance, or SEAD (suppression of enemy air defense), multiple UAVs need to arrive at a selected region from different directions. Each UAV plans its path dynamically considering the restriction including enemy radars, missiles, and its own performance. For the sake of the successful task achievement, it requires that all UAVs arrive at the goal position simultaneously or sequentially, which is called rendezvous problem.
In respect to the time coordination problem, the method using the coordination function and coordination variables is adopted by [1–4]. The optimal flight control sequence of the UAVs can be obtained to meet certain performance index by this method. It can not only simplify the complexity of solving the problem but also reduce the redundant communication links among the UAVs. But it is still a kind of centralized control method in the view of the command and control manner. After the UAVs flying into the mission area, the communication will be reduced or interrupted. In order to ensure the survivability and increase the probability of the successful task, a distributed control method based on multiagent average consensus algorithm is proposed by [5–9]. It can deal with the case of battlefield environment changes. Due to the adoption of the average consensus algorithm, it is a “compromise” collaborative result. Therefore, the consensus-based flight control sequence is not considered in an optimal manner. To improve the performance of consensus algorithm, literature [10] proposes a noncooperative based optimal consensus (NCOC) algorithm, which can be employed to solve this rendezvous problem. However, the overall cost of multi-UAV system is usually not the minimum, since these UAVs only minimize their own cost function rather than the overall one. This is due to the fact that these UAVs are selfish or noncooperative. Literature [11, 12] focuses on the dynamic response and optimal cost by introducing the outdated state difference and the optimized weighted matrix to the consensus algorithm. This method accelerates the speed of the consensus convergence. Similarly, the consensus algorithm with virtual leader and state predictor has been adopted in [13] to increase the convergence speed of the rendezvous problem.
Different from the existing methods using average consensus algorithm [5–9], this paper focuses on the optimal solution rather than the convergence speed [11–13]. Further, the NCOC method [10] is developed, and the cooperative method of solving the rendezvous problem is proposed. The system achieves the overall optimal cost in this process. The novel method can deal with the complex network restriction such as switching topologies, communication delay.
The remainder of this paper is organized as follows. Firstly, the mathematical description of rendezvous problem is given. Based on the results of [5–9], the solving framework is proposed using the coordination function and coordination variable. A novel distributed control method using the cooperative game based optimal consensus (CGOC) algorithm is proposed for the “cooperative” UAVs. Finally, numerical experiments and simulation results are illustrated to show the effectiveness and benefit of the proposed method.
2. Problem Description
2.1. Basic Assumptions and Physical Constraints
This paper focuses on the distributed control method of the rendezvous problem, rather than the flight control of the platform. Hence, we assume that(1)all UAVs are small size;(2)each UAV is equipped with the autopilot, which can track the waypoint automatically;(3)mission control and flight control can be decoupled, respectively. Thus, the flight control problem can be simplified. In order to show the physical characteristics of the platform, the flight constraints and related performance parameters are listed as follows: (i)the maximum and minimum speed , ;(ii)the minimum turning radius ;(iii)the maximum flight time .
2.2. Incomplete Kinematic Model of UAV
Similar to [1–7, 9], the 2D kinematic model of the UAV is selected to study the rendezvous problem ()where denotes the position vector of the th UAV and is the heading angle. is the velocity and is the changing rate of the heading angular velocity. and satisfyThe parameters , , and are determined by the physical performance.
The autopilot of each UAV maintains the expected heading angle and velocity. Its mathematical model can be described by two first-order differential equationsThe variable or vector with the superscript signifies the reference instruction. Parameters and are the constant coefficients of the heading and velocity channel of the autopilot.
2.3. Communication Model
Graph theory is used to describe the communication model among multi-UAV system; see, Figure 1. Let denote the relationship between multiple UAVs with the set of nodes , the set of edges , and adjacent matrix . The node indices belong to a finite index set . The edge can be depicted by , and the value of corresponds to the edge of the graph; that is, . The neighbors set of node is defined by .
Since the phenomenon such as communication delay, interruption, and limited bandwidth may appear in practical application, in this paper, two restrictions will be considered(i)communication delay : it denotes the transfer delay of the message from the th to the th UAV;(ii)time varying interconnections : denotes the switching signal with successive times to describe the topology switches.
2.4. Mathematical Model of Rendezvous Problem
The mathematical description of the rendezvous problem will be given as follows. The initial and target position of the th UAV are defined as and . Then, the flight trajectory can be defined asIf there exist input sequence and , satisfying , kinematic model (1), and restricts (2), (3), then is the feasible flight path of the th UAV.
For each feasible path, UAV should avoid threats and return to the base safely with adequate fuel. In fact, there is a strong coupling relationship among these restrictions. On one hand, avoiding threats means longer path and higher flight velocity. It demands less residence time of the aircraft in danger. On the other hand, fuel saving means shorter path and lower velocity. Thus, the rendezvous problem is a time coordination problem considering the exposure time in the mission area and fuel consumption.
Define the cost function of the path aswhere and . The threat cost is determined by the exposure time under the threat radar. If the signals emitted by the radar in all directions are the same, it is proportional to the fourth power of the distance between the UAV and the threat. In (6), is the position of the th threat, and is the threat set. Because the fuel consumption rate is determined by the air drag torque, the fuel cost is proportional to the square of the velocity. is the proportion factor.
With (5) and (6), the rendezvous problem is equivalent to the following global optimization equation:
Figure 2 demonstrates the rendezvous problem of three UAVs (the dashed line denotes the feasible path). For example, when number 2 UAV detects a new unknown threat, all UAVs need to negotiate with each other and determine a new ETA (estimated time of arrival). Then, a new path is replanned; see the solid line in Figure 2. The new path will ensure the UAVs arrive at their destination simultaneously. It is obvious that the costs of number 1 and number 3 UAV are not optimal for that path. But the bigger ETA negotiated will accomplish the mission. Therefore, rendezvous problem is a typical cooperative control problem of multi-UAV, including two aspects. One is path planning. Each UAV plans its path considering some constraints including radars, missile threats, and platform performance. The other one is trajectory control. Each UAV arrives at its destination simultaneously through adjusting its velocity and heading angle.
Obviously, there are also some challenges for solving the global optimization problem. Firstly, the expressions (1), (2), and (3) are noncomplete constraints of the motion model. So, it is difficult to generate the feasible path. Secondly, the gradient optimization technique is very sensitive to the initial path. Finally, time constraints mean that all UAVs should plan their path simultaneously. Therefore, the suboptimal or feasible solution of problem (7) is discussed.
3. Distributed Solution of the Rendezvous Problem
3.1. Distributed Solving Structure
3.1.1. Coordination Function and Coordination Variable
This method was proposed to reduce the communication cost and the difficulty of the problem by [1–4, 14]. Coordination variable is the minimum amount of information to coordinate multi-UAV system. Coordination function is the performance of the system achieving effective coordination. The basic idea is listed as follows [3].
For the th UAV, the state space description of the battlefield is defined as and the state as . The decision of each UAV would affect the overall cost. stands for the feasible decision set, and is the decision variable.
In order to achieve effective coordination, there is a minimum amount of information, namely, the coordination variable . If the variable and the corresponding function are known to each UAV, the system can achieve collaborative behavior.
3.1.2. Coordination Variable Selection
Distributed structure for solving multi-UAV rendezvous problem includes(i)Waypoint Planner (WP): obtain the waypoint sequence considering minimizing the threat and fuel consumption;(ii)Kinematic Trajectory Smoothing (TS): generate the fine trajectory in accordance with the kinematic model (1) and the UAV platform physical constraints (2), (3);(iii)Distributed Coordinator (DC): receive the ETA from its adjacent UAVs and adjust its ETA by consensus algorithm. The reference velocity instruction is generated according to the ETA. Get the reference heading instruction according to the trajectory point got from DS module. Send , to AP module;(iv)autopilot (AP): ensure the aircraft fly to the destination position according to and ;(v)kinematic model (DM): describe the kinematic characteristics of the platform. Assume that it satisfies formula (1).
Obviously, how to obtain the reference velocity is the key to DC module. Let denote the remaining path length to the prespecified target location of the th UAV. Then, . Given and , the ETA can be calculated as follows:
The total flight time of the th UAV isThe first derivative of the is calculated asThus, the reference velocity can be obtained
For this rendezvous problem, there are several advantages when the ETA time is selected as the coordination variable: (a) reducing the redundant information negotiating with each other, (b) lowering the difficulty of solving the optimization problem, (c) increasing the dynamic response capacity of multiple UAVs, and (d) cutting down the influence of the restricted networks on the solving structure.
4. Optimal Consensus Based on Cooperative Game
4.1. Cooperative Game Theory
Considering a team with players, the quadratic function (12) is constructed to describe the cost of the player in the team where and are positive definite matrices with proper dimension. Each player satisfies the following dynamical model:in which and are constant matrices with proper dimensions and is the decision of the th player. The state variable can be reflected by the other player’s decision in the minimization process. That means that the players may have conflicting interests. If a player decides to minimize its cost in a noncooperative manner, the decision chosen by the th player can increase the cost of the others due to the coupling relationship. However, if the players decide to cooperate, individual cost may be minimized and hence we can get a smaller team cost. This will result in the set of Pareto-efficient solutions. For the set of inequalities , if there is not at least one solution , is called Pareto-efficient solution and the corresponding costs are Pareto solution.
The solution of the minimization problem cannot be dominated by any other solutionwhere and . It is a set of Pareto-efficient solutions of the above problem. The solutions are the functions of the parameter . The final solution should be selected according to an axiomatic approach as our decision for the cooperative problem. The Nash-bargaining solution is selected in this paperwhere are the individual costs calculated by using the noncooperation solution that is obtained by minimizing the cost (12).
The coefficient can be obtained (see Theorem 6.10 in [15]) by
4.2. CGOC Problem Description
When the multiagent systems achieve consensus, all agents get the same values. Thus, the individual cost is defined aswhere and are symmetric positive definite matrices.
The overall cost of the multiagent systems is obtained by weighted individual costwhere the coefficient matrices are
According to cooperative game theory, the smallest overall cost can be obtained when the multiagent system achieve consensus. Taking the communication delay () into account, the dynamic model of the agent system can be described as follows:in which is the state vector. , , , and . is the solution of the Riccati equation (21), and is the element of the Laplacian matrix, which is used to describe the interconnection of the communication
The cooperative game based optimal consensus (CGOC) can be described as the following optimization problem:
According to cooperative game theory, the Pareto-optimal solution set can be obtained by
It is easy to see that Pareto-efficient solution is a function of the parameter . Further, the unique Nash bargaining solution is calculated from the above set. Considerin which is the individual cost calculated using the noncooperative strategy (see [10])
When the multiagent systems achieve consensus, the optimal parameter is determined. In the following, we will discuss how to get the optimal control strategy of problem (22).
4.3. CGOC Problem Solving
Due to the existence of the communication delay in the dynamic model (20) of the agent, it is difficult to get the exact solution or a numerical solution of such a problem. The solving method of the two-point boundary problem with delay and ahead term is adopted in this section [16].
Firstly, introduce the Lagrangian operator and define the Hamilton function
By the optimal control theory, we can getwhere , , and are the gradients at , , and . By formula (26), formulas (27) can be rewritten asin which the initial value is . Thus, the optimal control strategy is
Rewrite and as follows:
The sensitivity parameter is introduced; we haveand, here, is a scalar and it satisfies .
Assume that , , and are differentiable at , and their Maclaurin series converges at where , , and .
Thus, the suboptimal control strategy of the above problem (31) is equivalent to the sum of the solutions of the two-point boundary problem, including the 0th-order and the th-order problems.(i)Two-point boundary problem with the 0th-order(ii)Two-point boundary problem with the th-order
4.3.1. The Solution of the 0th-Order Problem
Assume that the Lagrangian operator iswhere is a matrix with proper dimension.
Substituting formula (34) with (39), we get
Moreover, the Lagrangian operator should satisfyOn the other hand, formula (35) can be rewritten as
Comparing (42) with (43), the Riccati matrices equation can be obtained as
Therefore, the optimal control strategy of the 0th-order problem can be obtained through calculating and .
4.3.2. The Solution of the th-Order Problem
Similar to the 0th-order problem, the Lagrangian operator is selected aswhere is a matrix with proper dimension and is an adjoint variable to be determined.
Substitute the Lagrangian operator into (36) and (38). We havewhereIn addition, the state equation with the th-order problem isin which the initial values are
Thus, the control strategy of the th-order problem isand the suboptimal control strategy is obtained by the former items
In (51), there exists an optimal control strategy when tends to , there exists an optimal control strategy. In particular, the control strategy is when .
In the following, the optimal algorithm of the CGOC problem (22) is given with proper Maclaurin items .
Algorithm 1.
Step 1. Solve the Riccati equation (44) to get .
Step 2. Get and by (41) and (39). Let .
Step 3. Calculate by (46).
Step 4. If , solve by (48). Otherwise, execute Step 6.
Step 5. Let , and then execute Step 3.
Step 6. Get the optimal control strategy by (51).
According to the cooperative game theory, the optimal control strategy is a function of the parameter . Thus, a recursive algorithm can be given to get the Nash bargaining solution.
Algorithm 2.
Step 1. Set the initial value .
Step 2. Calculate the suboptimal control strategy By Algorithm 1.
Step 3. Check the equation , . If it does not hold, there exists an satisfying . Update the parameter with the following formula, and then execute Step 2:
Step 4. Compute by (16).
Step 5. Update parameter byIf , hold, then terminate the algorithm and let . Otherwise, execute Step 2.
Next, we will discuss the convergence of the CGOC algorithm. When , the CGOC algorithm can be described asThe items and can be seen as the coefficients and in Theorem 2 () in [17]. Thus, the CGOC algorithm can converge with the given communication delay. When , the can also be rearranged with the items and . Similarly, the convergence range can be obtained.
The distributed solving method of the rendezvous problem is given in Algorithm 3.
Algorithm 3.
Step 1. Get the waypoints sequence of each UAV using the route planning method [18]. ConsiderStep 2. Calculate the path length in the WGS84 coordinate system and get the ETA ranges which satisfy the restrict (3). Define the set , . If holds, then execute Step 3. Otherwise, return to Step 1.
Step 3. Exchange the estimated arriving time with the other UAVs and confirm the communication delay , . Let .
Step 4. Get and by Algorithms 1 and 2. Then, obtain the optimal control strategy .
Step 5. Select the th component of , and substitute it into (11). The optimal reference velocity instruction can be obtainedStep 6. Calculate the reference heading instruction considering the waypoints information.
Step 7. Send the instructions and to the autopilot system (4). If the UAVs arrive at target position, terminate the algorithm; otherwise, execute Step 3.
5. Simulations and Results
5.1. Simulation Environment and Experimental Conditions
In order to validate the effectiveness of the proposed method, two experiments are illustrated in this section. A typical simulation environment of the multi-UAV rendezvous problem is given in Figure 3. Six artillery threats are placed in the square area (100 km 100 km). Let the initial positions of three UAV be (20 km, 20 km), (40 km, 10 km), and (60 km, 15 km), and let the target positions be (50 km, 70 km), (60 km, 60 km), and (70 km, 70 km). The results of the route planning are shown in the figure. The initial velocities of the three vehicles are 27 m/s 32 m/s and 30 m/s, respectively. The initial path lengths of the three UAVs are 64.96 km, 55.40 km, and 61.67 km using the TS module in the distributed solving structure. Their performance parameters are listed as follows: , , , , , and .
Two network restrictions are considered in the experiments. One is the time varying communication delay ; the other one is the time varying interconnection of the communication network:(1)time varying delay :(i)delay I: Markov random process with the probabilities 0.3, 0.3, and 0.4;(ii)delay II: normal distribution ;(iii)delay III: sine function ;(2)jointly connected topologies ; see Figure 4.
5.2. Experiment Results
In the simulation, we show the response of variable and velocity . Let . We can get the Nash bargaining solution by Algorithms 1 and 2. Further, the variable and the velocity can be obtained by Algorithm 3. The results are given in Figures 5, 6, and 7.
(a) Delay I
(b) Coordination variable
(c) Velocity variable
(a) Delay II
(b) Coordination variable
(c) Velocity variable
(a) Delay III
(b) Coordination variable
(c) Velocity variable
It is easy to see that the coordination variable achieves consensus from the Figures 5, 6, and 7. The time used to adjust the is less than 5 s. However, there is an obvious oscillation in that process. For the case of communication delays I, II, and III, the rendezvous times are 2103 s, 2106 s, and 2102 s, respectively. The velocity instruction can be computed by Algorithm 3, and their response cures are also shown in Figures 5(c), 6(c), and 7(c).
Furthermore, the simulation process of the rendezvous problem is shown in Figure 8. The result illustrates the effectiveness of the CGOC method.
(a) 242 s
(b) 576 s
(c) 913 s
(d) 1264 s
(e) 1790 s
(f) 2102 s
5.3. The Comparison of the Optimality and Dynamic Response
The comparison of the energy consumptions is given by using the proposed CGOC method and the NCOC method [10]. Considering the randomness and uncertainty property of the delay, 20 statistical experiments are given in Table 1 with communication delays I, II, and III.
The following conclusions can be obtained from Table 1.(1)The energy consumptions of the proposed CGOC method and the NCOC method are close. The results imply that the CGOC method has better robustness adaptability than NCOC method.(2)The overall cost using the CGOC method is less than that using the NCOC method. The case of number 3 UAV is the biggest one, about 41.7%. The second one is number 2 UAV, about 7.2%. Number 1 UAV is the minimum one, about 2.5%. In other words, the reduction of the overall cost is about 17.1%. Thus, the proposed CGOC method is better than NCOC in terms of the optimization.
Another important issue is the dynamic response when new incidents occur. For the case of the threat appearing (the length decrease 1.5 km) at 40 s and disappearing (the length increase 1.5 km) at 80 s, the dynamic responses are given in Figures 9 and 10. It is easy to see that CGOC requires less adjustment time and lower oscillation.
(a) Coordination variable
(b) Velocity variable
(a) Coordination variable
(b) Velocity variable
6. Conclusions
The cooperative game based optimal consensus (CGOC) algorithm has been proposed to solve the multi-UAV rendezvous problem with complex networks. Mainly, the following contributions have been concluded in this paper: (1) the mathematical model and distributed solving framework of the rendezvous problem have been established; (2) CGOC algorithm has been presented to minimize the overall cost of the multi-UAV system. The solving strategy of CGOC has been given theoretically using the cooperative game and sensitivity parameter method. Numerical examples and simulation results have been given to demonstrate the effectiveness with different network conditions and the benefit on the overall optimality and dynamic response.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This work was supported by NSFC (61203355) and STDP (20130522108JH) funded by China government.