Complexity

Volume 2017, Article ID 3813912, 12 pages

https://doi.org/10.1155/2017/3813912

## The Multiagent Planning Problem

^{1}Department of Fluid Mechanics, Faculty of Mechanical Engineering, Budapest University of Technology and Economics, Budapest, Hungary^{2}SPS Italiana Pack Systems, Novara, Italy

Correspondence should be addressed to Tamás Kalmár-Nagy; moc.yganramlak@ytixelpmoc

Received 31 July 2016; Revised 17 December 2016; Accepted 4 January 2017; Published 5 February 2017

Academic Editor: Roberto Natella

Copyright © 2017 Tamás Kalmár-Nagy et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

The classical Multiple Traveling Salesmen Problem is a well-studied optimization problem. Given a set of goals/targets and agents, the objective is to find round trips, such that each target is visited only once and by only one agent, and the total distance of these round trips is minimal. In this paper we describe the Multiagent Planning Problem, a variant of the classical Multiple Traveling Salesmen Problem: given a set of goals/targets and a team of agents, subtours (simple paths) are sought such that each target is visited only once and by only one agent. We optimize for minimum time rather than minimum total distance; therefore the objective is to find the Team Plan in which the longest subtour is as short as possible (a min–max problem). We propose an easy to implement Genetic Algorithm Inspired Descent (GAID) method which evolves a set of subtours using genetic operators. We benchmarked GAID against other evolutionary algorithms and heuristics. GAID outperformed the Ant Colony Optimization and the Modified Genetic Algorithm. Even though the heuristics specifically developed for Multiple Traveling Salesmen Problem (e.g., -split, bisection) outperformed GAID, these methods cannot solve the Multiagent Planning Problem. GAID proved to be much better than an open-source Matlab Multiple Traveling Salesmen Problem solver.

#### 1. Introduction

Applications from space exploration [1–3] and drone delivery to search and rescue problems [4–7] have underlined the need to plan a coordinated strategy for a team of vehicles to visit targets. It is important to develop a multiagent planner for a team of autonomous vehicles to cooperatively explore their environment [8, 9]. We formulate the overall planning problem as finding a near-optimal set of paths that allows the team of agents to visit a given number of targets in the shortest amount of time. This problem is quite similar to the well-known Multiple Traveling Salesmen Problem (MTSP) [10–13], a generalization of the Traveling Salesman Problem (TSP) [14–16], that can be stated as follows: given nodes (targets) and salesmen (agents), the MTSP consists of finding closed tours (paths start and end at the starting point of the agents), such that each target is visited only once and by only one agent and the total cost of visiting all nodes is minimal. MTSP has been used for modeling many real situations, from scheduling activities of companies and industries to cooperative planning problems. See, for example, [17], where MTSP is used for modeling the preprinted insert scheduling problem. Planning problems have also been investigated through MTSP formulations, specifically in [18, 19], where a dynamic mission planning system for multiple mobile robots operating in unstructured environments is presented (analysis of planetary exploration), or in [20], where the MTSP formulation is used to describe a path planning problem for a team of cooperative vehicles. An important and well-studied extension of the MTSP is the Vehicle Routing Problem [21, 22], where a fleet of vehicles of different capacities, based at either one or several depots, must deliver different customer demands (the number of vehicles is often considered as a minimization criterion in addition to total traveled distance).

A long-term goal of this work is to endow a team of autonomous agents (drones) with the capability of cooperative motion planning. In this application, the time available for the solution is limited and real-time algorithms providing good approximate solutions are required. In this work, the problem of planning a set of strategies for cooperatively exploring the environment with a fleet of vehicles is modeled as a variant of the classical MTSP, referred to as the Multiagent Planning Problem (MAPP): given nodes (targets) and salesmen (agents) located at different depots, the MAPP seeks tours such that each target is visited only once and by only one agent that minimizes a given cost function (specified by (8) and (9)). The paper presents a Genetic Algorithm Inspired Descent (GAID) method for obtaining good quality MAPP solutions.

The paper is organized as follows: after describing the MAPP in detail (Section 2) and an overview of how similar problems (MTSP in particular) are solved (Section 3), the proposed GA-Inspired Descent method is described in Section 4. Results are reported in Section 5, and conclusions are drawn in Section 6.

#### 2. The Multiagent Planning Problem: Notations

Graph theory [23, 24] provides a natural framework to describe the Multiagent Planning Problem. Given , a set of elements referred to as* vertices* (targets), and , a set of* edges* connecting vertices and , a* graph * is defined as the pair . Here we consider only* undirected graphs*, that is, graphs whose edges are unordered pairs with the symmetry relation . A* complete* graph is a graph where all vertices of are connected to each other. The complete graph induced by the vertex set is denoted by , where ( denotes the cardinality of a set) is the number of vertices. A graph is a* subgraph* of () if and such thatA subgraph is called a* path* in if is a set of vertices of the original graph andis the set of edges that connect those vertices. In other words, a path is a sequence of edges with each consecutive pair of edges having a vertex in common. Similarly, a subgraph of , withis called a* cycle*. The length of a path or cycle is the number of its edges. The set of all paths and cycles of length in will be denoted by and , respectively.

Paths and cycles with no repeated vertices are called* simple*. A simple path/cycle that includes every vertex of the graph is known as a Hamiltonian path/cycle. Graph is called* weighted* if a* weight* (or* cost*) is assigned to every edge . A weighted graph is called* symmetric* if . In this paper, the weight associated with each edge is the Euclidean distance between the corresponding vertices (locations); that is, The total cost of a path is the sum of its edge weightsAnalogously, for a cycle

After having introduced the necessary notation, we are now in the position to formalize the combinatorial problems of interest. The* Subtour Problem* [25] is defined as finding a simple path of length , starting at vertex and having the lowest cost . If , the problem is equivalent to finding the “cheapest” Hamiltonian path, where all the targets in are to be visited. The general Traveling Salesman Problem, or -TSP, poses to find a simple cycle of minimal cost starting and ending at vertex , visiting targets.

Let be the set of targets (goals) to be visited. The th target is an object located in Euclidean space whose position is specified by the vector . Let denote the set of agents with position specified by . The classical Multiple Traveling Salesman Problem can be formulated as follows. Let denote the unique depot; that is, . The* augmented vertex set* is and the configuration space of the problem is the complete graph . Let denote a cycle of length starting and ending at vertex (the depot). The Multiple Traveling Salesmen Problem is to find cycles (we also refer to these as* tours*) of length such that each target is visited only once (this also implies visitation by only one agent) and the sum of the lengths (costs) of all the toursis minimal.

The Multiagent Planning Problem can be formulated similarly. For the th agent, the augmented vertex set is given by and the configuration space of the problem is the complete graph . The weight associated with each edge is the Euclidean distance between the corresponding locations, that is, , with , rendering a weighted and symmetric graph. Let denote a simple path (no repeated vertices) of length starting at vertex . The optimal solution of the Multiagent Planning Problem (MAPP) is a set of pairwise disjoint (modulo the starting points) paths such that the length of the longest pathis minimal (a min–max problem). In other words, the agents have to visit targets in the least amount of time, and every target is visited only once. The number of targets visited by each agent can be different, but each agent has to visit at least one target (i.e., ).

#### 3. Overview of Solution Methods

The Multiple Traveling Salesmen Problem (MTSP) and the Multiagent Planning Problem are notoriously difficult to solve due to their combinatorial nature (they are NP-hard).

A common approach is to transform the MTSP into an equivalent Traveling Salesman Problem, for which solutions can be found by exact methods (e.g., branch-and-bound algorithms and linear programming [26–28]) or approximate algorithms such as Genetic Algorithms, Simulated Annealing, and Ant System [29, 30]. For example, in [31, 32] the authors proposed to transform the MTSP into an equivalent TSP by adding dummy cities and edges with ad hoc null or infinite costs. However, as stated in [33–35], transforming the MTSP into an equivalent TSP might yield an even harder problem to solve. Similar approaches are investigated in [13, 36, 37].

The first attempt to solve large-scale MTSPs is given in [33], where a branch-and-bound method (the most widely adopted technique for solving these combinatorial problems [38]) is applied to both Euclidean (up to cities and salesmen) and non-Euclidean problems (up to cities and salesmen). Branch-and-bound is also applied in [39] for solving an asymmetric MTSP up to cities.

Other solution methods have also been proposed, for example, simulated annealing [18], Gravitational Emulation Local Search (GELS) algorithm [40], and evolutionary algorithms. In [41], different evolutionary algorithms, ranging from Genetic Algorithms to Particle Swarm and Monte-Carlo optimization, are compared. The MTSP with ability constraint is solved with an Ant Colony Optimization (ACO) algorithm in [34], where the MTSP is not translated into an equivalent TSP and the ACO algorithm is modified for dealing with the characteristics of the original problem. In [34] results are compared with a Modified Genetic Algorithm that solves the equivalent TSP. Linear Programming is used in [35], and similarly to [34], the original MTSP is analyzed and solved. In both [34, 35], the authors conclude that the original MTSP is easier to solve than the derived TSP. An important work is [42], where different local search heuristics are presented and compared. In [17, 29, 43, 44] Genetic Algorithms are used to minimize the sum of the salesmen path lengths together with the maximum distance traveled by each salesmen (to balance the agent workload). A Modified Genetic Algorithm on the equivalent TSP is used in [32]. Shirafkan et al. [45] proposed a hybrid method incorporating simulated annealing and Genetic Algorithm to solve the multidepot MTSP. In this MTSP variant the starting location (depot) of each agent is fixed and multiple agents can have start from the same depot. The multiobjective MTSP variant is presented in [46] and Genetic Algorithm is used to find approximate solutions. In multiobjective MTSP the total distance traveled by the agents and the balance of agent working times are the objectives. Kaliaperumal et al. [47] also used Genetic Algorithm to solve the MTSP. They developed a brand new crossover operator called modified two-part chromosome crossover and obtained better results using this new operator. A new (branch-and-cut type) exact method is presented in [48] for the heterogeneous MTSP (some targets can be visited only by a specific agent). In [49] a heuristic search method is proposed to solve the Multiagent Path Finding problem, which is similar to MAPP, except that the endpoints of the tours are also fixed. These methods work well on the problem they were constructed for, but none of them is designed to solve the MAPP. In Section 5.1.2 we benchmark our GAID against some of these methods in MTSP solving. In Section 5.2 we compare GAID against a Matlab solver which is capable of solving MAPP as well as MTSP.

#### 4. The Multiagent Planning Problem: Approximate Solution

Given agents and known targets/cities to visit, the optimal team strategy (called Team Plan) is sought that allows the fleet to visit every target only once.

We represent the Team Plan as a collection of distinct subtours. Thus, given agents and targets, Team Plan is defined as , where is the path of the th agent visiting targets. Note that this representation allows the individual subtours to have different lengths. Moreover, the Multiagent Planning Problem can also be rewritten for each th agent to find the best possible subtour of length that satisfies the imposed cost function. We propose an optimization technique we call the Genetic Algorithm Inspired Descent (GAID) method. Briefly, a Genetic Algorithm (GA) is an optimization technique used to find approximate solutions of optimization and search problems [50–52]. Genetic Algorithms are a particular class of evolutionary methods that use techniques inspired by Darwin’s theory of evolution and evolutionary biology, such as inheritance, mutation, selection, and crossover. In these systems, populations of solutions compete and only the fittest survive.

Similarly to the classical GA, GAID consists of two phases: initialization and evolution. In the initialization phase, the starting Team Plan is created (see Section 4.1), while the evolution phase (see Section 4.2) evolves Team Plan toward a better quality final solution.

##### 4.1. Initialization Phase

During the initialization phase, the starting Team Plan—a starting set of subtours—is created. Let and be the sets of targets and the agents, respectively. Without loss of generality, we assume that the order of planning is and that the starting subtours have similar lengths. A Team Plan is feasible if the subtours are pairwise disjoint (except possibly their starting points).

At first, a subtour (with starting point ) of length is chosen. Then subtour is chosen from the target set that is simply obtained by discarding from the targets visited by agent . In general, the th agent plans a subtour on the targets not yet allocated previously. Obviously, this process yields a feasible Team Plan .

Here we utilize three simple initialization methods.(i)Random initialization: for each agent the targets are selected at random from the unallocated ones.(ii)Greedy initialization: the initial Team Plan is created using a greedy approach to form feasible starting subtours. Each agent selects its targets using a Nearest Neighbor heuristics: for a given a target, the next target will be the nearest one.(iii)TSP-based initialization: for problems where the positions of the agents are not imposed (the agents can start from any target ), a feasible starting Team Plan can be generated by clustering the TSP solution computed on the complete graph . “Clustering” is carried out by discarding edges from the TSP tour, in order to subdivide it “fairly”, and having starting subtours with similar costs. This initialization method introduces a degree of complexity in the overall system, since a TSP solution must be computed.

The initialization phase is an important step in the optimization process, since it directly influences the efficacy of the algorithm.

##### 4.2. Evolution Phase

The evolution phase evolves the Team Plan, trying to design a strategy where the overall time is reduced (minimizing the cost ; see (9)).

This phase has the same mechanism of a classical Genetic Algorithm [50] with one important difference: there is only one Team Plan to be evolved (i.e., the population size is 1).

At every evolution/generation step, a set of operators (see Section 4.3) is applied to the subtours . If improves, it is kept for the next generation step, otherwise the previous Team Plan is restored (we expect that a simulated annealing-type modification will be even more efficient). The flowchart of the GAID evolution phase is shown in Figure 1.