Abstract

To reduce the impact of uncertainties caused by unknown motion parameters on searching plan of moving targets and improve the efficiency of UAV’s searching, a novel distributed Multi-UAVs cooperative search control method for moving target is proposed in this paper. Based on detection results of onboard sensors, target probability map is updated using Bayesian theory. A Gaussian distribution of target transition probability density function is introduced to calculate prediction probability of moving target existence, and then target probability map can be further updated in real-time. A performance index function combining with target cost, environment cost, and cooperative cost is constructed, and the cooperative searching problem can be transformed into a central optimization problem. To improve computational efficiency, the distributed model predictive control method is presented, and thus the control command of each UAV can be obtained. The simulation results have verified that the proposed method can avoid the blindness of UAV searching better and improve overall efficiency of the team effectively.

1. Introduction

In recent years, Unmanned Aerial Vehicles (UAVs) have been widely used in both civilian and military fields, performing a variety of missions such as searching, reconnaissance, and surveillance [1]. Carrying a series of sensors, they can search kinds of targets in mission areas and obtain large amounts of useful information. Currently, searching method is generally preplanning trajectory method, and it is effective to search steady and nonthreat targets. However, with the complexity of the environment and increase of threats, it will become very difficult for a single UAV to accomplish the tasks of searching and capturing. It may be destroyed by the enemies, and then the entire mission will be in failure. To solve this problem, many scholars have proposed cooperative search method by multiple UAVs [24]. Multi-UAVs cooperative search is precondition performing other tasks such as target tracking, attack, and evaluation. Efficient searching can find the targets in mission areas and obtain intelligence information. Then, how to realize better cooperation between UAVs has become an important problem to be resolved.

For the problem of Multi-UAV cooperative search, there are many literatures developed by associated scholars. In [5, 6], a cooperative search method based on distributed model predictive control (DMPC) is presented. Based on this method, a centralized online optimization problem is decomposed into decentralized optimization problem of each UAV. In [7], the objective of teams is proposed to search and cover the whole of the unknown area, while avoiding collision. Another problem is communication delay between UAVs when carrying on cooperative search in an uncertain environment [8, 9]. In [8], a stochastic method is proposed to estimate the probability of different actions for each UAV, and it can effectively compensate the effect of communication delay. For searching static targets, -shortest path algorithm is introduced to guide the UAVs in an unknown area. Compared with stochastic and greedy searching algorithm, it is verified to be more effective [10]. In [11], the distributed approach integrating dynamic programming is proposed, and the objective is to gain more information of the whole team. By this method, each UAV can find its own flight path, respectively.

Although these above methods have been verified to be effective for cooperative searching problems, they are mainly about the static targets, not suitable for the moving and time-critical targets. Currently, searching moving targets is the main focus of the satellites and robots fields [1218]. Here several typical results are listed. In [12], five methods are proposed to search the ocean moving targets, and their performances are compared by simulation experiments according to creditable orbit data of the satellite. In [14], a swarm of UAV is used to search one or more evading targets in a predefined area. These targets have the abilities of avoiding the detection of the UAV. By negotiation between UAVs, the swarm can optimize their sensing capabilities to ensure the maximal territory of searching. In [16], for the uncertainty of target motion, the distributed target-centric formation control strategy integrating sliding mode control is proposed. It can compensate the uncertainties and maintain the expected target-centric formation.

In this paper, we focus on searching the moving targets using Multi-UAV cooperation. Our main contributions contain three aspects: Cognitive information map is constructed to describe the interaction among target, environment, and UAVs. A novel updated mechanism for cognitive information map is designed integrating the prediction of target motion. The pheromone map and artificial potential field are combined into the cooperative cost, which can ensure effective cooperation and collision avoidance between UAVs.

The remainder of this paper is organized as follow. In Section 2, the problem description is presented. The model of cognitive information map is built in Section 3. Based on these efforts, the performance function of UAV searching is constructed, and the receding horizon optimization method is used to solve distributed cooperative decision-making problem in Section 4. The simulation results are shown in Section 5. In Section 6, we conclude this paper and point out the further direction of this work.

2. Problem Descriptions

2.1. The Description of Search Environment

As is mentioned above, UAVs detect the ground moving targets by using onboard sensors with a fixed angle. So the whole environment can be mapped into a two-dimension space. Given a mission area , suppose field of view (FOV) of the sensor to be approximately as a square area. Using geometric method, the mission area can be divided into same grids. Let the coordinate denote the position of one grid, where and . It is supposed that each grid can store some useful information, containing the existing probability of target, the uncertainty degree of environment, and the situation. Combining with all the grids, the cognitive information map (CIM) for searching can be obtained.

To better describe the searching problem, three hypotheses are given as follows: There is only a target in each grid at the same time. Communication between UAVs has no delay or interruption. There are no threats and obstacles in the environment, and only take into account the collision problem between UAVs.

2.2. Simplified Dynamic Model of UAV

Assume that each UAV has a certain speed and orientation. Let and denote the position and flight orientation of UAV , respectively. There are eight possible flight orientations, shown in such an array , 1 , , , , , , . The UAV’s dynamics is subjected by its curvature radius constraints. That means it can only change its orientation at most once by one step, namely, . Each UAV has three possible orientations in the next time step, such as turn left, go straight, and turn right. It means that UAV’s maximum turning angle is . For each UAV, the decision-making is which path will be selected in the next time step. Take UAV , for example; let denote its selected path at each time step . Figure 1 shows the schematic diagram for searching environment and various orientations.

As is shown in Figure 1, it is composed of two schematic diagrams. The left one is the description of UAV’s searching environment. The red triangle denotes UAV, the blue circle denotes target, and the black rectangle region denotes a preassumed river or lake, where there is a lower probability or even zero. The green rectangle region denotes a camp or an interesting target region, where there is a higher probability. The right one shows the possible flight orientations of UAV at the current and the next time step. If UAV flies towards the north at time , there will be . For each UAV, there are three possible selections for flight orientations at time , which are (towards northwest), (towards north), and (towards northeast), respectively. But only one orientation can be selected, which should be most useful for searching based on acquired external environment information.

3. The Model of Cognitive Information Map

Traditional existing probability map (TEPM) constructed in Section 3.1 only considers target existing probability in the task area, but it is not enough for Multi-UAVs cooperative search moving target. Next, the other two maps are introduced on the basis of TEPM. One is called the uncertain map of environment (UEM), and it can describe the certain degree of unknown environment which has been visited by UAVs. The other is called the pheromone map (PM), which is mainly used to establish the cooperation mechanism between UAVs and the controllable accessing ability to environment. So, integrating TPM, UEM, and PM, the cognitive information map (CIM) can be designed.

3.1. The Uncertain Map of Environment

For each grid cell, a number is set, which denotes uncertain degree of environment. For example, let denote uncertain degree of grid at time , and let denote descent factor. When UAV flies across the cell and understands grid cell, uncertain degree of grid cell will decrease. Set a certain threshold for the uncertain degree, and it means UAV completely understands the environment. Using the following equation, uncertain degree of grid cell can be updated:

3.2. Pheromone Map

In order to realize the effective cooperation between UAVs, two basic pheromones [19] which are attractant and repulsion are introduced in this section. For these two pheromones, two mechanisms are introduced, namely, propagation and evaporation. Define and as the coefficients of attractant and repulsion propagation, respectively, while defining and as the coefficients of attractant and repulsion evaporation, respectively. Similarly, define two same pheromones for each grid. Let and denote attractant and repulsion pheromone of grid at time , respectively.

Assume that is the last accessed time of the grid cell , is time threshold which is accessed again, and is switch coefficient of attractant pheromone of grid cell . If , then , which means grid is not be accessed again. If , then , which means grid should be accessed again. The switch matrix of attractant pheromone is calculated as follows:

Assume that is the accessed state matrix, storing the state of each grid accessed. Let denote the times of UAV’s accessing grid in the last period, where . Consider

Then, the attractant strength of the current grid can be calculated by (5), shown as follows:where denotes attractant strength of the grid at time , denotes the release amount of attractant pheromone at time , and denotes attractant pheromone from neighboring grid between times and .

Similar to attractant pheromone, the repulsion pheromone of current grid can be calculated by (6). Considerwhere denotes repulsion strength of the grid at time , denotes the release amount of repulsion pheromone at time , and denotes repulsion pheromone from neighboring grids between times and .

To ensure effective cooperation between UAVs, the total pheromone strength of the grid at time can be obtained by the following equation:

3.3. The Existing Probability Map of Target

If prior intelligent information obtained is not accurate, UAV cannot absolutely know the target distribution states. Existing probability map [11] is used to describe target existing probability of each grid. Let denote target existing probability of grid at time , let denote sensor detection precision, let denote sensor false probability, and let denote whether the target is detected or not at time. If , it means UAV has detected target, while means that UAVs do not detect target. Using the onboard sensors to detect all the grid cells, target existing probability is updated according to the detecting results. The updated mechanism is as follows:

To reduce the uncertainty of target motion parameters and improve the searching efficiency, prediction model of targets motion is established in this section. Then, a Gaussian distribution transition probability density function of moving target is proposed. Based on updating target existing probability in the TEPM, the existing prediction probability of the target can be calculated to further update the TEPM. So it can avoid the searching blindness and improve cognition and prediction ability for the target.

3.3.1. The Description of Potential Movement Area of Maneuvering Target

If the prior intelligent information and the previous uncertain motion of maneuvering target manners are not accurate, it will be very difficult for UAV to search maneuvering target. Therefore, it is very important to predict the motion manners of target. Here, the impact on these constraints is considered. The potential movement area of maneuvering target is built, shown in Figure 4. Assume that UAV has acquired prior information at time and enters search area after . During this time, target is moving. Suppose, at time , the position of maneuvering target is , denotes velocity of target, denotes heading angle of target, is the maximum heading angle, and is the maximum velocity. The potential motion area of maneuvering target in time is described as such a region, where is center, and are radius, and the shadow between heading angle and is target potential area, shown in Figure 2(a). If prior information is deficient, there will be and . It means velocity and heading angle of target are unknown; then the potential movement area of maneuvering target can approximate a cirque, shown in Figure 2(b).

3.3.2. The Motion Prediction Model of Maneuvering Target

In this paper, the probability distribution of maneuvering targets in potential movement area is used to predict the motion manners of target. Assume and denote position component of target on the axis of and at time , respectively. Similarly, and denote position component of target on the axis of and at time . Here, three hypotheses are given. The position component of target on the axis of and is independent. There is a larger time interval between and , and denotes the number of time step, where . The acceleration is the same in each time step and subject to Gaussian white noise sequence with the same variance.

According to the above description, mathematical induction method can be applied to gain the position and velocity of the target at subsection between and time, respectively. Consider

According to formulation (7)–(10), suppose . In this paper, is selected. Then, the position and velocity of the target towards - and -axis at time can be obtained as follows:

3.3.3. The Transition Probability Density Function of Maneuvering Target

The uncertainty of target position is mainly determined by the moving direction of the target. After , the circle with the center and the radius is formed. The transition probability density function of maneuvering target can be obtained. It is subject to a Gaussian distribution, the mean values of which are the points on the cycle mentioned above. The schematic of transition probability density function of maneuvering target is shown in Figure 3.

Assume that the position of target is at time , after , and the position of target along - and -axis will be such Gaussian distributions with the means and , variance , respectively. According to formulation (13), solving the mean and variance of random variable , it can be deduced as follows:

Combining (17) with (16), the variance of the random variable can be obtained:

Assume that the estimation of moving velocity is and the angle is between the velocity direction and -axis, and then (15) can be transformed as follows:

Similarly, one can obtain the mean and variance of random variable :

Assume random variables and are independent. After , the target moves from to , and then transition probability density function will be

3.3.4. The Existence Prediction Probability of Maneuvering Target

For ground moving target, UAV continuously needs to detect, update, and predict the moving of target and makes use of predictive result to update the existence prediction probability of the target. The predictive result will be the prior information for UAV’s decision-making at next time step.

Assume that the probability of the target existing in the grid at time is . The length and width of the grid are both . After , UAV will predict the moving situation of targets and gain the transition probability density function of the target moving from grid to grid . Consider

The existence prediction probability of targets in the grid at time is shown as follows:

At last, the cognitive information map (CIM) integrating uncertain map of environment, probability map of target, and pheromone map is completely built. It can provide more information to UAVs and ensure the effective cooperation between Multi-UAV.

4. The Decision-Making Approach of Distributed Cooperative Search of Multi-UAVs

In order to ensure effective cooperation and avoid collision between UAVs, the special mechanism based on the combination of pheromone map and artificial potential field method is established. Meanwhile, the correlative value function is constructed. But, for centralized model prediction control, calculation amounts are very large. If the central UAV is disabled, the other UAVs will not acquire decision-making commands; thus the whole searching task will be failed. So the centralized online optimization problem of Multi-UAVs is transformed into the distributed online optimization problem. It can improve the calculation efficiency and ensure successful accomplishment of the searching task.

4.1. The Performance Index Function of UAV Searching

The purpose of cooperative search is to find as many targets as possible and to reduce the uncertain degree of the whole searching area, thus ensuring the effective cooperation between UAVs and avoiding collision [19, 20]. We can construct the function of searching efficiency to describe performance of UAV searching. This function is a multiobjectives optimization function, containing coverage, target discovery, and cooperative capability. Next, we will introduce the modeling process of the search cost of environment, the discovery cost of target, and cooperative cost in detail.

(1) The Search Cost of Environment. The search cost of environment is mainly used to describe how to reduce the uncertainty degree of environment. The information entropy is introduced to denote the uncertainty degree of environment, presented as . The special definition is described as follows:

With UAV continuously searching and detecting in the mission area, the information entropy may gradually decrease. So, in order to describe the reduced amount of information entropy, the search cost of environment is defined as follows:

(2) The Discovery Cost of Target. The discovery cost of target denotes the possibility of UAV finding targets based on onboard sensors from the current position to target position:where is updated by formulation (2), is the detection precision of the sensor, and is false probability of the sensor.

(3) Cooperative Cost. The purpose of cooperation optimization is to avoid the excessive repeated searching of one grid. Therefore, the pheromone is introduced to describe the occupation situation of searching area. By this way, it can effectively reduce the numbers of UAV visiting the grid repeatedly and improve the cooperative searching efficiency. The cooperative cost is defined as follows:

4.2. Collision Avoidance Mechanism between Multi-UAVs

For Multi-UAVs cooperative search problem, we not only consider cooperative search efficiency but also consider the collision and trajectory overlap between UAVs. Here, the artificial potential field method is introduced to solve the above problem. This method is a virtual method and can ensure the relative position and orientation by calculating competition force between UAVs. Let denote the competition force between UAV and UAV at time , which is defined as follows:where denotes the shortest distance between the flight path of UAV and UAV , if , the constant is changed to a larger number, and if is gradually rising, is gradually dropping.

To facilitate the description, two conditions are given.

Condition 1. The distance between the UAV and UAV is less than the maximum distance , and the UAV lies in flight observation scope of UAV , where the scope is a sector area which is between the maximum angle and the minimum angle .

Condition 2. The difference of heading angle between the UAV and UAV is less than the threshold ; that is, .

The condition and the vector of competition force between UAVs are shown in Figure 6. From Figure 4, it can be found that UAV 3 and UAV 4 satisfy Condition 1, both in the flight range of UAV 1 and less than the maximum distance. UAV 2 and UAV 3 satisfy Condition 2, the angle between UAVs is less than , so only UAV 3 satisfies Conditions 1 and 2 at the same time, and only UAV 3 produces competition force for UAV 1; others do not produce competition force for UAV 1; that is, , , and .

The resultant force is the vector sum of competition forces which belong to the other UAVs act on the UAV . Consider

As is mentioned above, both UAV’s repeated access and collision avoidance between UAVs should be considered, so we will model competition force value function, shown as follows:where denotes angle difference between the direction of total rivaling force and the direction of each possible path. is a positive constant.

Combining (28) and (31), the cooperation value function can better express cooperative mechanism between UAVs. It can both improve the efficiency of cooperative search and avoid collision between UAVs:

The total value function is obtained by weighting every performance function:

4.3. The Decision-Making Approach of Distributed Cooperative Search

The purpose of Multi-UAV cooperative search moving targets is to ensure maximizing search efficiency in uncertain area, obtaining the information of the whole search area, decreasing the uncertain degree of environment, and finding the maximum number of targets.

Assume that the discrete states equation of Multi-UAVs cooperative search system is shown as follows:where is the discrete time, is the system states, is the control input, is the mapping from input to output, and is the length of predictive horizon. Here, and contain the states and controls of all UAVs, shown as follows:

Based on the current state and decision-making sequence during the horizon time, predictive states sequence can be obtained. Assume that and are the sets of the states and decision-making sequence, respectively, shown as follows:

Here, and denote the state and control sequence of UAV , respectively. Assume that is the performance function of UAV; the total performance function of UAV autonomous cooperative search during the prediction horizon can be written as

Then, the model of centralized prediction control of cooperative search at time can be obtained as follows:where and denote the feasible state set and admissible input set of UAV, respectively, denotes state constraint set of UAV , and denotes the various constraint conditions.

For centralized optimization model, the central node (UAV or ground station) is responsible for obtaining the state information of all UAVs and assigning the optimization control sequence to each UAV. This method has a good ability of global decision-making, but it may bring severe dimension disaster when the numbers of UAV and prediction horizon time increase. But, for each UAV, it must make the decision in a short time. So distributed model predictive control (DMPC) [20] method is introduced to realize decision-making of UAV searching. The information structure of UAV autonomous cooperative search based on DMPC is shown in Figure 5.

For such a dynamic decoupling system, assume that is the state equation of UAV . UAV can obtain information by the network communication and make the decision by the DMPC method. It also can share the state and the decision with other neighboring grids. For the UAV  , the local horizon optimization decision-making model of autonomous cooperative search can be written asThe distributed decision-making algorithm of Multi-UAVs cooperative search can be divided into steps, shown in Algorithm 1.

Initialize: Assume that the prediction horizon length is , sample time is , discrete time is .
 At time , , is initialized. Predict the states of each UAV and obtain prediction
 sequences , , .
Step  1. Let the first item of control sequence as the command of output decision.
Step  2. Update the system states , and predict the states and decision sequences of
 UAV as , , .
Step  3. UAV sends its state and decision-making information to UAV , and receives the
 prediction states sequence , of the other UAVs.
Step  4. Compute the value of each value function based on formulation (26), (27), (28), (31), and (32), and obtain
 total cost value of UAV in the prediction horizon length .
Step  5. According to formulation (39), make the local decision and obtain decision-making sequence
, define as decision-making output at time .
Step  6. Update , return to Step  1.
End

5. The Simulation Validation and Results Analysis

To verify the effectiveness and feasibility of the proposed method, the simulation platform of Multi-UAVs distributed cooperative search for moving target is established in this section. The dynamic flight trajectory of UAV and motion trajectory of target can be observed in the simulation platform. The conditions of the simulation are designed as follows. The size of task area is 80 km × 80 km and divided into grids with the same width and length = 1 km. Ten stochastic moving targets are distributed in the task area. Three UAVs take off from one base and enter the bottom edge of the search area after a short time. Assume that the speed of UAV is 30~50 m/s, the speed of moving target is 3~5 m/s, and the turning angle constraints of UAV and target are ~ and ~, respectively. The simulation results are shown in Figures 6 and 7.

Because the process of UAV searching is dynamic, different steps are selected to describe searching behavior of UAV and the coverage of environment area. In Figure 7, the thin line denotes the search trajectory of UAV and the rectangle denotes the moving trajectory of target. As is shown in Figure 7(a), UAVs can detect some targets at the beginning. With the simulation ongoing, UAVs can continually access individual cells, know about the task area, and confirm whether the target exists. From Figure 7(d), three UAVs have already covered most of the task area and detected more targets than that in Figure 7(a). Figure 8 shows the potential difference variation of pheromone. There are no overlaps of trajectory between the UAVs and no resources to be wasted. The proposed method can avoid collisions and ensure better cooperation between UAVs. All the results above have verified the effectiveness and feasibility of the proposed method.

To further verify the effectiveness of the proposed method in this paper, it is compared with cooperative search method with nonprediction, scanning beam method, and greedy search method, respectively. There are two aspects of comparison, namely, average numbers of finding targets and average coverage ratio for the task area. Because the initial positions of UAV and moving targets, along with the initialization of area environment, are stochastic in the simulation, twenty simulations are performed, where the length of simulation step is set at 1000. The simulation results are shown in Figures 8 and 9, respectively.

According to Figure 8, the average numbers of finding targets for four methods are almost the same at beginning. But, with the increasing simulation steps, it can be found that the proposed method in this paper can effectively find more moving targets in the task area than the other three methods. That has verified the effectiveness of the proposed method on the number of finding targets. From Figure 9, it can be seen that the final average coverage ratio of the proposed method is 44.5%, better than greedy search method and nonprediction cooperation method and a little inferior to scanning beam search method.

6. Conclusions

As a result of the restriction of the single UAV platform, it might not detect and find moving targets in the complex environment and is even destroyed by enemy units, causing the failure of the entire search and reconnaissance mission. So a distributed cooperative search decision-making method of Multi-UAV based on the moving prediction of targets is proposed in this paper. In this method, detection information of onboard sensors is used to update target existing probability based on Bayesian theory. A Gaussian distribution of target transition probability density function is introduced to calculate prediction probability of moving target existence, and then target probability map can be further updated in real-time. We will build the performance function of cooperative search and adopt receding horizon optimization method to solve and find the maximum control sequence in which searching performance function is to maximize and lead each UAV to effectively search in the task area. The simulation result shows that the proposed method can reduce the uncertainty and blindness in the process of UAV searching through introducing moving prediction of target and improve the entire searching performance of UAV and observes the flight trajectory of Multi-UAV in real-time.

Future research may focus on the design of cognitive information map (CIM) and solving the optimization solution of distributed cooperative search and control. The more reasonable CIM will be designed to obtain more information, including environment, obstacles, threat, and targets. Searching and capturing intelligent evasion targets is also an important problem to be further researched.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgment

The research is supported by the National Aviation Science Foundation of China (NASFC) under Grant no. 20135896027.