Abstract

The classical Multiple Traveling Salesmen Problem is a well-studied optimization problem. Given a set of goals/targets and agents, the objective is to find round trips, such that each target is visited only once and by only one agent, and the total distance of these round trips is minimal. In this paper we describe the Multiagent Planning Problem, a variant of the classical Multiple Traveling Salesmen Problem: given a set of goals/targets and a team of agents, subtours (simple paths) are sought such that each target is visited only once and by only one agent. We optimize for minimum time rather than minimum total distance; therefore the objective is to find the Team Plan in which the longest subtour is as short as possible (a min–max problem). We propose an easy to implement Genetic Algorithm Inspired Descent (GAID) method which evolves a set of subtours using genetic operators. We benchmarked GAID against other evolutionary algorithms and heuristics. GAID outperformed the Ant Colony Optimization and the Modified Genetic Algorithm. Even though the heuristics specifically developed for Multiple Traveling Salesmen Problem (e.g., -split, bisection) outperformed GAID, these methods cannot solve the Multiagent Planning Problem. GAID proved to be much better than an open-source Matlab Multiple Traveling Salesmen Problem solver.

1. Introduction

Applications from space exploration [13] and drone delivery to search and rescue problems [47] have underlined the need to plan a coordinated strategy for a team of vehicles to visit targets. It is important to develop a multiagent planner for a team of autonomous vehicles to cooperatively explore their environment [8, 9]. We formulate the overall planning problem as finding a near-optimal set of paths that allows the team of agents to visit a given number of targets in the shortest amount of time. This problem is quite similar to the well-known Multiple Traveling Salesmen Problem (MTSP) [1013], a generalization of the Traveling Salesman Problem (TSP) [1416], that can be stated as follows: given nodes (targets) and salesmen (agents), the MTSP consists of finding closed tours (paths start and end at the starting point of the agents), such that each target is visited only once and by only one agent and the total cost of visiting all nodes is minimal. MTSP has been used for modeling many real situations, from scheduling activities of companies and industries to cooperative planning problems. See, for example, [17], where MTSP is used for modeling the preprinted insert scheduling problem. Planning problems have also been investigated through MTSP formulations, specifically in [18, 19], where a dynamic mission planning system for multiple mobile robots operating in unstructured environments is presented (analysis of planetary exploration), or in [20], where the MTSP formulation is used to describe a path planning problem for a team of cooperative vehicles. An important and well-studied extension of the MTSP is the Vehicle Routing Problem [21, 22], where a fleet of vehicles of different capacities, based at either one or several depots, must deliver different customer demands (the number of vehicles is often considered as a minimization criterion in addition to total traveled distance).

A long-term goal of this work is to endow a team of autonomous agents (drones) with the capability of cooperative motion planning. In this application, the time available for the solution is limited and real-time algorithms providing good approximate solutions are required. In this work, the problem of planning a set of strategies for cooperatively exploring the environment with a fleet of vehicles is modeled as a variant of the classical MTSP, referred to as the Multiagent Planning Problem (MAPP): given nodes (targets) and salesmen (agents) located at different depots, the MAPP seeks tours such that each target is visited only once and by only one agent that minimizes a given cost function (specified by (8) and (9)). The paper presents a Genetic Algorithm Inspired Descent (GAID) method for obtaining good quality MAPP solutions.

The paper is organized as follows: after describing the MAPP in detail (Section 2) and an overview of how similar problems (MTSP in particular) are solved (Section 3), the proposed GA-Inspired Descent method is described in Section 4. Results are reported in Section 5, and conclusions are drawn in Section 6.

2. The Multiagent Planning Problem: Notations

Graph theory [23, 24] provides a natural framework to describe the Multiagent Planning Problem. Given , a set of elements referred to as vertices (targets), and , a set of edges connecting vertices and , a graph is defined as the pair . Here we consider only undirected graphs, that is, graphs whose edges are unordered pairs with the symmetry relation . A complete graph is a graph where all vertices of are connected to each other. The complete graph induced by the vertex set is denoted by , where ( denotes the cardinality of a set) is the number of vertices. A graph is a subgraph of () if and such thatA subgraph is called a path in if is a set of vertices of the original graph andis the set of edges that connect those vertices. In other words, a path is a sequence of edges with each consecutive pair of edges having a vertex in common. Similarly, a subgraph of , withis called a cycle. The length of a path or cycle is the number of its edges. The set of all paths and cycles of length in will be denoted by and , respectively.

Paths and cycles with no repeated vertices are called simple. A simple path/cycle that includes every vertex of the graph is known as a Hamiltonian path/cycle. Graph is called weighted if a weight (or cost) is assigned to every edge . A weighted graph is called symmetric if . In this paper, the weight associated with each edge is the Euclidean distance between the corresponding vertices (locations); that is, The total cost of a path is the sum of its edge weightsAnalogously, for a cycle

After having introduced the necessary notation, we are now in the position to formalize the combinatorial problems of interest. The Subtour Problem [25] is defined as finding a simple path of length , starting at vertex and having the lowest cost . If , the problem is equivalent to finding the “cheapest” Hamiltonian path, where all the targets in are to be visited. The general Traveling Salesman Problem, or -TSP, poses to find a simple cycle of minimal cost starting and ending at vertex , visiting targets.

Let be the set of targets (goals) to be visited. The th target is an object located in Euclidean space whose position is specified by the vector . Let denote the set of agents with position specified by . The classical Multiple Traveling Salesman Problem can be formulated as follows. Let denote the unique depot; that is, . The augmented vertex set is and the configuration space of the problem is the complete graph . Let denote a cycle of length starting and ending at vertex (the depot). The Multiple Traveling Salesmen Problem is to find cycles (we also refer to these as tours) of length such that each target is visited only once (this also implies visitation by only one agent) and the sum of the lengths (costs) of all the toursis minimal.

The Multiagent Planning Problem can be formulated similarly. For the th agent, the augmented vertex set is given by and the configuration space of the problem is the complete graph . The weight associated with each edge is the Euclidean distance between the corresponding locations, that is, , with , rendering a weighted and symmetric graph. Let denote a simple path (no repeated vertices) of length starting at vertex . The optimal solution of the Multiagent Planning Problem (MAPP) is a set of pairwise disjoint (modulo the starting points) paths such that the length of the longest pathis minimal (a min–max problem). In other words, the agents have to visit targets in the least amount of time, and every target is visited only once. The number of targets visited by each agent can be different, but each agent has to visit at least one target (i.e., ).

3. Overview of Solution Methods

The Multiple Traveling Salesmen Problem (MTSP) and the Multiagent Planning Problem are notoriously difficult to solve due to their combinatorial nature (they are NP-hard).

A common approach is to transform the MTSP into an equivalent Traveling Salesman Problem, for which solutions can be found by exact methods (e.g., branch-and-bound algorithms and linear programming [2628]) or approximate algorithms such as Genetic Algorithms, Simulated Annealing, and Ant System [29, 30]. For example, in [31, 32] the authors proposed to transform the MTSP into an equivalent TSP by adding dummy cities and edges with ad hoc null or infinite costs. However, as stated in [3335], transforming the MTSP into an equivalent TSP might yield an even harder problem to solve. Similar approaches are investigated in [13, 36, 37].

The first attempt to solve large-scale MTSPs is given in [33], where a branch-and-bound method (the most widely adopted technique for solving these combinatorial problems [38]) is applied to both Euclidean (up to cities and salesmen) and non-Euclidean problems (up to cities and salesmen). Branch-and-bound is also applied in [39] for solving an asymmetric MTSP up to cities.

Other solution methods have also been proposed, for example, simulated annealing [18], Gravitational Emulation Local Search (GELS) algorithm [40], and evolutionary algorithms. In [41], different evolutionary algorithms, ranging from Genetic Algorithms to Particle Swarm and Monte-Carlo optimization, are compared. The MTSP with ability constraint is solved with an Ant Colony Optimization (ACO) algorithm in [34], where the MTSP is not translated into an equivalent TSP and the ACO algorithm is modified for dealing with the characteristics of the original problem. In [34] results are compared with a Modified Genetic Algorithm that solves the equivalent TSP. Linear Programming is used in [35], and similarly to [34], the original MTSP is analyzed and solved. In both [34, 35], the authors conclude that the original MTSP is easier to solve than the derived TSP. An important work is [42], where different local search heuristics are presented and compared. In [17, 29, 43, 44] Genetic Algorithms are used to minimize the sum of the salesmen path lengths together with the maximum distance traveled by each salesmen (to balance the agent workload). A Modified Genetic Algorithm on the equivalent TSP is used in [32]. Shirafkan et al. [45] proposed a hybrid method incorporating simulated annealing and Genetic Algorithm to solve the multidepot MTSP. In this MTSP variant the starting location (depot) of each agent is fixed and multiple agents can have start from the same depot. The multiobjective MTSP variant is presented in [46] and Genetic Algorithm is used to find approximate solutions. In multiobjective MTSP the total distance traveled by the agents and the balance of agent working times are the objectives. Kaliaperumal et al. [47] also used Genetic Algorithm to solve the MTSP. They developed a brand new crossover operator called modified two-part chromosome crossover and obtained better results using this new operator. A new (branch-and-cut type) exact method is presented in [48] for the heterogeneous MTSP (some targets can be visited only by a specific agent). In [49] a heuristic search method is proposed to solve the Multiagent Path Finding problem, which is similar to MAPP, except that the endpoints of the tours are also fixed. These methods work well on the problem they were constructed for, but none of them is designed to solve the MAPP. In Section 5.1.2 we benchmark our GAID against some of these methods in MTSP solving. In Section 5.2 we compare GAID against a Matlab solver which is capable of solving MAPP as well as MTSP.

4. The Multiagent Planning Problem: Approximate Solution

Given agents and known targets/cities to visit, the optimal team strategy (called Team Plan) is sought that allows the fleet to visit every target only once.

We represent the Team Plan as a collection of distinct subtours. Thus, given agents and targets, Team Plan is defined as , where is the path of the th agent visiting targets. Note that this representation allows the individual subtours to have different lengths. Moreover, the Multiagent Planning Problem can also be rewritten for each th agent to find the best possible subtour of length that satisfies the imposed cost function. We propose an optimization technique we call the Genetic Algorithm Inspired Descent (GAID) method. Briefly, a Genetic Algorithm (GA) is an optimization technique used to find approximate solutions of optimization and search problems [5052]. Genetic Algorithms are a particular class of evolutionary methods that use techniques inspired by Darwin’s theory of evolution and evolutionary biology, such as inheritance, mutation, selection, and crossover. In these systems, populations of solutions compete and only the fittest survive.

Similarly to the classical GA, GAID consists of two phases: initialization and evolution. In the initialization phase, the starting Team Plan is created (see Section 4.1), while the evolution phase (see Section 4.2) evolves Team Plan toward a better quality final solution.

4.1. Initialization Phase

During the initialization phase, the starting Team Plan—a starting set of subtours—is created. Let and be the sets of targets and the agents, respectively. Without loss of generality, we assume that the order of planning is and that the starting subtours have similar lengths. A Team Plan is feasible if the subtours are pairwise disjoint (except possibly their starting points).

At first, a subtour (with starting point ) of length is chosen. Then subtour is chosen from the target set that is simply obtained by discarding from the targets visited by agent . In general, the th agent plans a subtour on the targets not yet allocated previously. Obviously, this process yields a feasible Team Plan .

Here we utilize three simple initialization methods.(i)Random initialization: for each agent the targets are selected at random from the unallocated ones.(ii)Greedy initialization: the initial Team Plan is created using a greedy approach to form feasible starting subtours. Each agent selects its targets using a Nearest Neighbor heuristics: for a given a target, the next target will be the nearest one.(iii)TSP-based initialization: for problems where the positions of the agents are not imposed (the agents can start from any target ), a feasible starting Team Plan can be generated by clustering the TSP solution computed on the complete graph . “Clustering” is carried out by discarding edges from the TSP tour, in order to subdivide it “fairly”, and having starting subtours with similar costs. This initialization method introduces a degree of complexity in the overall system, since a TSP solution must be computed.

The initialization phase is an important step in the optimization process, since it directly influences the efficacy of the algorithm.

4.2. Evolution Phase

The evolution phase evolves the Team Plan, trying to design a strategy where the overall time is reduced (minimizing the cost ; see (9)).

This phase has the same mechanism of a classical Genetic Algorithm [50] with one important difference: there is only one Team Plan to be evolved (i.e., the population size is 1).

At every evolution/generation step, a set of operators (see Section 4.3) is applied to the subtours . If improves, it is kept for the next generation step, otherwise the previous Team Plan is restored (we expect that a simulated annealing-type modification will be even more efficient). The flowchart of the GAID evolution phase is shown in Figure 1.

4.3. Team Plan Operators and Boosting

The evolution of toward a near-optimal multiagent strategy is accomplished by combining the genetic materials of its subtours through the application of genetic-like operators. Three different operators have been designed: crossover, mutation, and migration. The operators are applied in a predefined order, and their application depends on a given probability, as shown in Figure 1.

The crossover operator (Figure 2) is applied with probability and combines the genetic materials of two selected subtours (called parents), replacing them with the two newly created ones (the offsprings). Parents are chosen using the best-worst selection with probability (Figure 2(a)) or randomly (Figure 2(b)) with probability . The parents are mated the classical way [50]: they are randomly halved (not necessarily at the same position) and the halves are simply swapped.

The Mutation Operator changes the Team Plan (with probability ) by randomly swapping two genes between two different subtours. The Migration Operator (applied with probability ) removes a randomly chosen target from subtour (of length ) and adds it to subtour (of length ). The location at which this target is placed into subtour is also chosen at random. Note that the lengths of the subtours change: decreases while increases.

With probability , each subtour is processed with a heuristic boosting technique. The 2-opt method [5355] is adopted here to directly improve the quality of subtours (it replaces subtours with better ones from their “neighborhood”). The 2-opt method determines whether the inequalitybetween the four vertices , , , and of a path/subtour holds, in which case edges and are replaced with edges and , respectively. This method provides a shorter path without intersecting edges.

5. Results

An extensive set of simulations were run to test the performance of the proposed GAID method. Unless otherwise specified, simulations were run for generation steps, while the crossover, mutation, migration, and boosting (2-opt) operators were applied with probabilities , , , and , respectively. For the application of the crossover operator, the probability of best-worst selection is (thus the probability for random selection is ).

5.1. Comparison with Structured and Known Solutions

In this section, GAID is compared with known results. In order to make the comparison meaningful, we also imposed the same constraints (if any) used in the referenced works.

5.1.1. An Example Problem with Known Solution

In this section we present a 9-81 MAPP (9 agents, 81 targets), in which the agents and targets form tightly packed clusters as shown in Figure 3. Each cluster contains an agent and 9 targets. Since the distances among the clusters are considerably longer than those among the points of a cluster, the optimal solution can be found by treating each cluster as a 1-9 MAPP (1 agent, 9 targets). The tour lengths are between 356.1 and 548.6, meaning that the optimal cost (the length of the longest tour) is (see (9)). In Figure 3 we show the random and greedy initialization for this MAPP. In both cases some of the subtours contain a higher number of targets than others. We intentionally set the initialization this way to further challenge the algorithm. The initial tour lengths vary between 1944.9–15302.3 for the random, and 175.3–1627.9 for the greedy initialization. Consequently, the initial Team Plan costs are and , respectively.

The convergence of the Team Plan cost for both simulations is shown in Figure 4 together with the cost of the optimal solution (). At the end of the simulations the costs of the subtours varied between 378.3–612.3 and 389.9–594.1 for solutions starting from randomly initialized and greedy initialized Team Plans, respectively.

The final Team Plans of the two simulations are shown in Figure 5. We see that those long edges which initially connected targets belonging to different clusters were completely eliminated. The longest subtour connects the targets in the middle-top cluster for both simulations.

5.1.2. Comparison with Evolutionary Algorithms

We also compare our method with the results reported in [34], where an Ant Colony Optimization algorithm is compared with the Modified Genetic Algorithm (MGA) of [32]. We note here that these methods cannot solve the MAPP, therefore six MTSPs, all taken from well-known TSPLIB [56], are solved. The number of agents is set to , and their starting location is the first target in the corresponding problem data file. Since a single depot is considered, only the greedy initialization method is used. In both [32, 34], rounded costs are considered. Therefore, for meaningful comparison, we need to minimize the following cost function:

In addition, the maximum length of the cycles, , is limited to 20 (note that since MTSPs are considered here, we need to modify the Team Plan using cycles instead of paths). Results based on simulations are presented in Table 1. In this case, our results outperform those reported in the above references. Figure 6 also shows the solution obtained for the pr76 TSPLIB problem.

5.1.3. Comparison with Heuristics

We compare our method against the results reported in [42], where different heuristics for solving MTSP instances are proposed and compared.

In [42], a no-depot MTSP variant is considered (the agents do not have a predefined starting location), and the min–max optimization problem is solved ((9) is minimized). In addition, the number of salesmen is fixed and no constraints on the plan lengths are imposed. Note that since we are comparing our method with MTSP results, we need to modify our Team Plan accordingly, using cycles instead of paths. In addition, since in [42] rounded distances are considered, for meaningful comparison the cost function (see (9)) is modified accordingly: For each test case, our results are compared with only the best ones of [42]. We also report the name of the Heuristic with which each referenced solution has been obtained (please refer to [42] for a more detailed description of the adopted heuristic methods). Tables 2 and 3 show the results obtained by initializing the Team Plan with either the greedy or the TSP-based initialization methods, respectively. Results are averaged over simulations.

In each test case, GAID returns good solutions, regardless of the applied initialization method. In general, our best solutions are close to the ones from the literature and in a few cases (e.g., in the berlin52 or the pr264 problems) are even better. The greedy initialization method seems to provide a better initial Team Plan in those problems where the distribution of targets has a well-defined structure (the pr264 problem), while in “small-size” problems (i.e., berlin52) the TSP-based method is preferable (even if the obtained improvement does not justify its required complexity, time, and computational costs). Even though these heuristics are superior in solving MTSP compared to GAID, they cannot solve MAPP, while our method can solve both.

Figure 7 shows the solutions obtained by applying the greedy (Figure 7(a)) and the TSP-based (Figure 7(b)) initialization methods for the kroA100 problem (with agents). Compared with [42], the TSP-based method and the greedy method result in final solutions that are only and worse than the cited one, respectively.

5.2. Comparison with Other Software

In this section we test GAID against a freely available Matlab MTSP solver based on a Genetic Algorithm [57].

The number of agents is fixed, and the minimum path size, , is imposed (this way, solutions with paths composed of only one target are avoided). The cost function, which is specified by (9), is minimized, and targets are randomly generated over the unit square map. In all simulations, the greedy initialization method is adopted. Since the Matlab MTSP solver is based only on a Genetic Algorithm and no heuristics is used, we also run a set of tests with .

Table 4 shows the obtained results, averaged over simulations. Our method clearly outperforms the Matlab solver (clearly, with the 2-opt method the results are much better). Figure 8 shows two examples where the Matlab MTSP solver solution is compared with the GAID ones.

5.3. Solution for a Large Example Problem

We tested our algorithm on a 20-400 MAPP. The initial Team Plan was constructed by means of the greedy initialization method, such that each subtour visits targets. The simulation was ran for generations.

Figures 9 and 10 summarize the results. The initial cost of was reduced by % (to ) by the end of the th generation and by a further % (to ) till the end of the simulation.

6. Conclusions and Future Work

This paper describes the Multiagent Planning Problem, a variant of the classical Multiple Traveling Salesman Problem. Our solution method uses a simplified Genetic Algorithm, called Genetic Algorithm Inspired Descent (GAID), capable of solving both the MTSP and the MAPP. GAID takes an initial Team Plan and applies various genetic operators to decrease the length of the longest path. We benchmarked GAID against other evolutionary algorithms and heuristics. GAID outperformed the Ant Colony Optimization and the Modified Genetic Algorithm. Even though the heuristics specifically developed for MTSP (e.g., -split, bisection) outperformed GAID, these methods cannot solve MAPP. GAID proved to be much better than the Matlab MAPP solver. GAID is also easy to implement.

A long-term goal of this work is to endow a team of autonomous agents (drones) with the capability of cooperative motion planning. In this application, the time available for the solution is limited and fast algorithms providing good approximate solutions prevail. The results presented here show the success of the approach, demonstrating how a simple method can solve an otherwise hard combinatorial problem.

Competing Interests

The authors declare that they have no competing interests.

Acknowledgments

This work relates to the scientific programs “Development of quality-oriented and harmonized R+D+I strategy and the functional model at BME” (Project ID: TÁMOP-4.2.1/B-09/1/KMR-2010-0002) and “Talent care and cultivation in the scientific workshops of BME” (Project ID: TÁMOP-4.2.2/B-10/1-2010-0009).