An Optimal Cooperative Control Design for State Consensus of Periodic Multiagent Systems

Ayatollahi, Mehrasa; Majd, Vahid Johari; Nasrabadi, Mohammad Mehdi

doi:https://doi.org/10.1155/2018/3183278

Mathematical Problems in Engineering

On this page

Abstract Introduction Preliminaries Numerical Example Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2018 | Article ID 3183278 | https://doi.org/10.1155/2018/3183278

An Optimal Cooperative Control Design for State Consensus of Periodic Multiagent Systems

Mehrasa Ayatollahi,¹Vahid Johari Majd,²and Mohammad Mehdi Nasrabadi¹

Academic Editor: Zoran Gajic

Received04 Apr 2018

Accepted25 Jul 2018

Published06 Aug 2018

Abstract

The problem of cooperative optimal control of multiagent systems with linear periodic continuous-time dynamics is considered. The state consensus problem is formulated as an optimal control problem in which the consensus requirement is reflected in the cost. The cost optimization of each subsystem is considered over finite horizon while the states of the agents converge to a common value, with a control signal that depends on the interactions of the neighboring subsystems. The proposed control law consists of a local and regional terms to capture local measurements and measurements due to interactions with the neighboring agents, respectively. These two terms are obtained by solving a Hamilton-Jacobi-Bellman partial differential equation. A numerical example is presented to demonstrate the effectiveness of the proposed method.

1. Introduction

In recent years, there has been a growing interest towards coordination control of a group of dynamical agents with prescribed dynamics. Multiagent systems, which consist of agents and their environments, are considered in many applications such as multi-robot systems [1–3], unmanned aerial vehicles [4–7], formation control [8–12], and swarms [13–16] to help accomplish tasks that cannot be completed by individual ones.

Various dynamics can be considered for the agents in a multiagent team. Among these, multiagent systems with linear periodic dynamics that may be employed in several applications such as satellite networks or robotic systems, are considerable. The importance of the study of periodic control systems is that the properties of periodicity may be used to achieve more suitable design, and its capability allows better describing physical dynamics with cyclic behavior such as Low Earth Orbit (LEO) satellites or robotic systems [17–19]. Furthermore, the theory of periodic systems provides useful tools to improve the control performance of the closed-loop systems and also is adequate enough for solving the problems of time-invariant systems where time-invariant controllers are inadequate [20, 21].

One of the first works on periodic multiagent systems was presented [22] for discrete-time dynamical systems by extending a gradient based optimization approach of a single periodic system to a multiagent case. The method considers the network’s cost as a linear combination of agent’s costs and minimizes it by finding a single periodic output feedback. A stable optimal state feedback is proposed in [23] to control a group of periodic agents that can move at different speeds but only in the same direction. However, the designed control signal for each agent is considered completely independent from other agents’ feedback.

Consensus problem is a fundamental topic in the area of cooperative control of multiagent systems. In a network of agents, consensus means to reach an agreement (either in state or output) regarding a certain quantity of interest while the agents have different initial states [24]. Various control methods such as optimal control, event-based control, and sliding mode control have been investigated so far for the consensus problem of multiagent systems (see [25, 26] and the references therein). In [27], the consensus problem is solved via a periodically intermittent control for a class of second-order agents while the optimality condition is not considered. In [28], a networked system whose agents can only communicate with their local neighbors is considered and a decentralized event-based control strategy is proposed for the consensus problem without considering the optimality conditions. In [29], an optimal controller is designed to achieve state consensus for a team of agents with discrete-time dynamics.

A closed-loop solution for optimal control problems can be achieved by solving HJB equation if one can guess a suitable solution candidate [30–32]. Among studies on the optimization of multiagent networks based on the HJB equation, HJB equations in coupled quadratic forms with nonquadratic input energy cost have been formed in [33] to investigate the optimal synchronization control for generic linear multiagent systems with input saturation. Authors in [34] focused on interactions among the agents to design a controller for a multiagent team in continuous-time case to reach output consensus. The result was implemented on a group of mobile robot vehicles. However, for the existence of a solution to HJB equation, the method requires nonsingular square input matrices. A similar method is incorporated in [35] to reach an optimum state consensus controller for a team of nonlinear agents. Authors in [36] designed an optimal consensus based formation controller for a team of mobile robots via HJB equation by considering both full connectivity and partial connectivity for their assumed network.

The main focus of this paper is on designing an optimum group cooperation in the sense of optimal consensus for a team of agents with continuous-time periodic dynamics. We assume that each agent of the team is connected to some other agents known as neighbors such that all the agents are in connection with each other directly or indirectly, such that the connectivity of the entire network is guaranteed. Having only a limited access to the information from the neighbors, the agents have to exchange information to achieve cooperation. Considering this challenge, we design a distributed control system with a separate controller for each agent that uses partial information of its neighbors. Each agent in the proposed team has its own cost function that must be minimized while the main goal of the team is to reach state consensus. Unlike the design in [22], we design distributed controllers for the agents to minimize their costs according to the feedback signals received from neighbors and also their own information. Similar to the method incorporated in [37], we assume that the control signal consists of two parts, but, unlike [34], in our proposed design method both parts are in feedback form. By the connectivity assumption, to reach state consensus, it suffices to consider state consensus for each agent only with its neighbors. The consensus requirement is reflected in the cost function of each agent. Similar to [34], we use the general method of HJB for solving the minimization problem, but, unlike that work, the input matrices do not need to be invertible. In addition, we incorporate interval numbers for the first time for periodic systems in order to extend a result of LTI systems to periodic systems.

The paper is organized as follows: Section 2 provides the preliminary background and the problem definition. In Section 3, our strategy to reach the optimal control signal is explained and the proposed design method is offered. In Section 4, a numerical example and the simulation results are provided. Finally, the conclusion remarks are given in Section 5.

2. The Problem Definition and Preliminaries

A multiagent team consists of a set of agents , where denotes the number of agents in the team. Similar to [32, 38, 39] all agents are assumed to have the same dynamical model, which is typical in many applications. Each agent id considered a linear periodic continuous-time system of the following form:where and are state and input vectors, respectively, and and are periodic matrices with period , i.e., and .

The overall system can be written as follows:where and , and , and , , and .

For cooperation and coordination among the team’s agents, each agent must be aware of the status of other agents, which in this work is considered as their states. Hence, the agents need to communicate with each other through two-way links. It is assumed that the network is partially connected such that there is no isolated agent in the team and that all agents are connected to each other either directly or indirectly via their neighboring sets. The connection among the neighboring agents is defined by an nonsingular and symmetric matrix known as neighboring matrix with and other entries as follows:

In this work, we consider the problem of optimal state consensus for a team of periodic agents; i.e., all agents in a neighboring set must reach to the same state. For reaching state consensus suppose that one arbitrary agent approaches the desired state; then any other agent for which can directly reach the common reference state too. For an agent such that connectivity assumption implies the existence of an information path between and through their neighbors which helps to reach the desired value. Considering the mentioned hypothesis, the main aim of this work is to minimize each individual agent cost function via its local and regional feedback signals such that all agents in the team reach the state consensus.

According to what is mentioned above, we define the cost function for agent by importing state consensus condition as a part of cost, by the following expression:where matrix is positive semi-definite, is a symmetric positive definite matrix, and is a symmetric positive definite matrix.

3. Main Results

In this section, we are going to propose distinct control laws to minimize cost functions (4) for the periodic multiagent system (1). To this end, we are going to use HJB partial differential equations [40, 41].

For a team of agents with dynamical equations introduced by (1) and costs introduced by (4), the corresponding HJB equation for agent is as follows:where is a value function which must be chosen such that the partial differential equation (5) satisfies with the boundary condition .

Due to the connectivity assumption, each agent of the introduced model receives information from its neighboring agents and passes its own information to them through existing communications links. As mentioned before, the control law for each agent consists of a local feedback as well as regional feedback signals received from its neighbors. Thus, we can decompose the control signal of agent into the following two parts:where is the local feedback matrix and (for ) are regional feedback matrices corresponding to the neighboring agents. We also consider the following theorem to construct the required feedback matrices such that consensus is guaranteed.

Lemma 1 (see [42]). There exists a state feedback such that the system (2) achieves consensus if and only if there exist a positive definite matrix and a matrix such thatwhere is an orthonormal matrix in , for some , and is its orthonormal complement. Then the control law can be reconstructed by .

Remark 2. The entries of a periodic matrix in one period can be expressed as interval numbers, which are a set of real numbers between two numbers: [43]. Although Lemma 1 was originally given for time-invariant systems, using the calculus of interval numbers and following the same derivations given in the proof of Lemma 1, one can easily yield the same result for the periodic systems.
The following theorem gives and that leads to the optimal control law for minimization of cost function (4), while state consensus is achieved.

Theorem 3. Assume a team of periodic agents whose dynamics are governed by (1) with controllable pair , and, with the entire system dynamics of given by (2), and the cost functions are governed by (4). For the agent , (for ), consider the control law (6) with local feedback matrix,and the neighboring feedback matrices,in which is an positive definite symmetric matrix with continuously differentiable entries computed from:and (for ) are positive definite matrices with continuously differentiable entries computed from:then the proposed control protocol (6) will minimize the cost (4) while state consensus is achieved.

Proof. Consider a continuously differentiable value function candidate of the following form:that should satisfy the HJB equation (5). Here, , , and are matrices with continuously differentiable entries where is symmetric positive definite and is positive definite. Substituting (12) into (5) yields:To carry out the minimization on the right-hand side, we let:which yields where the first term is the local control signal and the second term is the regional counterpart. To show that the resulting minimizes the objective function, we consider the second derivative of the right-hand side of (14) with respect to as follows:According to our assumptions, is a positive definite matrix, and, thus, the resulting in (15) is the optimum control signal.
To determine and , one can substitute (15) into (13), which leads to the following ODEs:The last two ODEs must hold for all s.t. .
The first equation in (17) is a matrix differential Riccati equation. According to [44], positive definiteness of and and the controllability of guarantee the existence of the unique satisfying in the first equation. The two other differential equations in (17) are linear in and , respectively. Due to the theorem of existence and uniqueness of solutions to the linear first-order differential equations [45], the existence of unique solutions to two remaining equations in (17) is guaranteed by existence of . This implies that the HJB equation (5) has a solution in the form of (12) which satisfies the boundary conditions. This leads to the optimal solution.
Now consider presentation of entire system as (2); then the optimum control signal (15) in entire form is as follows:where , and s.t. for and for . Then the entire closed-loop form is with . Now let be an orthonormal basis for the nullspace of and be its orthonormal complement, i.e., and , then, for every , the third relation in (7) of Lemma 1 is satisfied. If it does not, then there exists such that:Left multiplying (19) by and right multiplying it by yield which is contradiction. Due to the definition of , matrix is invertible and . Thus, we have:Due to the definition of , we have or equivalently:As and are positive definite matrices, we can conclude that is also positive definite and invertible. Therefore, By choosing , and , the relation (21) can be represented as:Now by right multiplying (22) by and making use of (20) and noting that we obtain:Thus, the first relation in (7) satisfies. To verifying the second relation, let:then we have the entire closed-loop system as follows:or equivalently:where is the orthonormal complement of . For every , system converges to a point in and . Therefore, implies the eigenvalues of have negative real part. Thus, there exists a matrix such that:By choosing and substituting the above-mentioned and , the second relation in (7) is obtained and consensus is achieved.

4. Numerical Example

In this section, the proposed approach is applied to a multiagent team consisting of 3 identical SISO periodic subsystems. The matrices of each linear time periodic agent are given by [46] which are borrowed from an aeromechanic system:The neighboring matrix of this system is given by:

Required matrices for cost functions are taken as:

Appling ODEs (16) on the controllable pair leads to the following results:

Figure 1 that shows states of the system illustrates state consensus of the team for the arbitrary initial state , , over one period . Figures 2, 3, and 4 show differences between states of each of the two separate agents. These figures verify that the state differences converge to zero over one period which is equivalent to state consensus.

5. Conclusion

In this paper, the problem of cooperative control in a multiagent team with periodic dynamics was investigated and a control protocol to minimize agent’s individual costs subject to availability of some other agent’s information was introduced. The control law was obtained from solving HJB partial differential equation and consists of local and regional terms that are stated in feedback form. In addition, the states of all the agents in the team converge to a common reference value, and thus the state consensus is achieved. The effectiveness of the proposed method was demonstrated through an illustrative example.

Data Availability

The simulations are generated based on the given model in the paper and no external data is used for them.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

The authors would like to sincerely thank Dr. Mahdieh Tahmasebi for her useful discussions and helps throughout this work.

References

Y. Tang, X. Xing, H. R. Karimi, L. Kocarev, and J. Kurths, “Tracking Control of Networked Multi-Agent Systems under New Characterizations of Impulses and Its Applications in Robotic Systems,” IEEE Transactions on Industrial Electronics, vol. 63, no. 2, pp. 1299–1307, 2016.
View at: Publisher Site | Google Scholar
J. Jin, J. Ramirez, S. Wee, D. Lee, Y. Kim, and N. Gans, “A switched-system approach to formation control and heading consensus for multi-robot systems,” Intelligent Service Robotics, vol. 11, no. 2, pp. 207–224, 2018.
View at: Publisher Site | Google Scholar
Y. Song, C. Zhang, and H. Liu, “Method of quantitative analysis for multirobot cooperative hunting behaviors,” Mathematical Problems in Engineering, vol. 2016, Article ID 4370635, 7 pages, 2016.
View at: Publisher Site | Google Scholar
T. Han, M. Chi, Z.-H. Guan, B. Hu, J.-W. Xiao, and Y. Huang, “Distributed three-dimensional formation containment control of multiple unmanned aerial vehicle systems,” Asian Journal of Control, vol. 19, no. 3, pp. 1103–1113, 2017.
View at: Publisher Site | Google Scholar | MathSciNet
S. Moon, E. Oh, and D. H. Shim, “An integral framework of task assignment and path planning for multiple unmanned aerial vehicles in dynamic environments,” Journal of Intelligent & Robotic Systems, vol. 70, no. 1-4, pp. 303–313, 2013.
View at: Google Scholar
L. He, X. Sun, and Y. Lin, “Distributed output-feedback formation tracking control for unmanned aerial vehicles,” International Journal of Systems Science, vol. 47, no. 16, pp. 3919–3928, 2016.
View at: Publisher Site | Google Scholar | MathSciNet
P. A. Beardsley, S. F. Watson, and J. Alonso-Mora, “Shared control of semi-autonomous vehicles including collision avoidance in multi-agent scenarios,” U.S. Patent 9,216,745, issued December 22, 2015.
View at: Google Scholar
F. Xiao, L. Wang, J. Chen, and Y. Gao, “Finite-time formation control for multi-agent systems,” Automatica, vol. 45, no. 11, pp. 2605–2611, 2009.
View at: Publisher Site | Google Scholar | MathSciNet
A. Loria, J. Dasdemir, and N. Alvarez Jarquin, “Leader-Follower Formation and Tracking Control of Mobile Robots Along Straight Paths,” IEEE Transactions on Control Systems Technology, vol. 24, no. 2, pp. 727–732, 2016.
View at: Publisher Site | Google Scholar
X. Dong, Y. Zhou, Z. Ren, and Y. Zhong, “Time-varying formation control for unmanned aerial vehicles with switching interaction topologies,” Control Engineering Practice, vol. 46, pp. 26–36, 2016.
View at: Publisher Site | Google Scholar
X. Sun and C. G. Cassandras, “Optimal dynamic formation control of multi-agent systems in constrained environments,” Automatica, vol. 73, pp. 169–179, 2016.
View at: Publisher Site | Google Scholar | MathSciNet
X. Dong, Q. Tan, Q. Li, and Z. Ren, “Necessary and sufficient conditions for average formation tracking of second-order multi-agent systems with multiple leaders,” Journal of The Franklin Institute, vol. 354, no. 2, pp. 611–626, 2017.
View at: Publisher Site | Google Scholar | MathSciNet
D. Teodorović, “Transport modeling by multi-agent systems: A swarm intelligence approach,” Transportation Planning and Technology, vol. 26, no. 4, pp. 289–312, 2003.
View at: Publisher Site | Google Scholar
H. Shi, L. Wang, and T. Chu, “Swarming behavior of multi-agent systems,” Journal of Control Theory and Applications, vol. 2, no. 4, pp. 313–318, 2004.
View at: Publisher Site | Google Scholar | MathSciNet
S. Das, D. Goswami, S. Chatterjee, and S. Mukherjee, “Stability and chaos analysis of a novel swarm dynamics with applications to multi-agent systems,” Engineering Applications of Artificial Intelligence, vol. 30, pp. 189–198, 2014.
View at: Publisher Site | Google Scholar
H. Liang, H. Su, X. Wang, and M. Z. Q. Chen, “Swarming of heterogeneous multi-agent systems with periodically intermittent control,” Neurocomputing, vol. 207, pp. 213–219, 2016.
View at: Publisher Site | Google Scholar
M. M. Fateh, H. A. Tehrani, and S. M. Karbassi, “Repetitive control of electrically driven robot manipulators,” International Journal of Systems Science, vol. 44, no. 4, pp. 775–785, 2013.
View at: Publisher Site | Google Scholar | MathSciNet
N. Athanasopoulos, M. Lazar, C. Böhm, and F. Allgöwer, “On stability and stabilization of periodic discrete-time systems with an application to satellite attitude control,” Automatica, vol. 50, no. 12, pp. 3190–3196, 2014.
View at: Publisher Site | Google Scholar | MathSciNet
X. Gao, K. L. Teo, and G.-R. Duan, “An optimal control approach to spacecraft rendezvous on elliptical orbit,” Optimal Control Applications and Methods, vol. 36, no. 2, pp. 158–178, 2015.
View at: Publisher Site | Google Scholar | MathSciNet
B. Zhou and Z.-Y. Li, “Truncated predictor feedback for periodic linear systems with input delays with applications to the elliptical spacecraft rendezvous,” IEEE Transactions on Control Systems Technology, vol. 23, no. 6, pp. 2238–2250, 2015.
View at: Publisher Site | Google Scholar
S. Bittanti and P. Colaneri, Periodic systems: filtering and control, Springer Science Business Media, 2009.
A. Varga and S. Pieters, “Gradient-based approach to solve optimal periodic output feedback control problems,” Automatica, vol. 34, no. 4, pp. 477–481, 1998.
View at: Publisher Site | Google Scholar | MathSciNet
D. DeVon and T. Bretl, “Control of many robots moving in the same direction with different speeds: a decoupling approach,” in Proceedings of the American Control Conference, ACC'09, pp. 1794–1799, St. Louis, MO, USA, 2009.
View at: Google Scholar
R. Olfati-Saber, J. A. Fax, and R. M. Murray, “Consensus and cooperation in networked multi-agent systems,” Proceedings of the IEEE, vol. 95, no. 1, pp. 215–233, 2007.
View at: Publisher Site | Google Scholar
J. Qin, G. Zhang, W. X. Zheng, and Y. Kang, “Adaptive sliding mode consensus tracking for second-order nonlinear multiagent systems with actuator faults,” IEEE Transactions on Cybernetics, pp. 1–11, 2018.
View at: Publisher Site | Google Scholar
J. Qin, W. Fu, Y. Shi, H. Gao, and Y. Kang, “Leader-following practical cluster synchronization for networks of generic linear systems: an event-based approach,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–10, 2018.
View at: Publisher Site | Google Scholar
L. Xiao and X. Liao, “Periodic intermittent consensus of second-order agents networks with nonlinear dynamics,” International Journal of Control, Automation, and Systems, vol. 12, no. 1, pp. 23–28, 2014.
View at: Publisher Site | Google Scholar
X. Duosi, X. Yin, and J. Xie, “Decentralized event-based communication strategy on leader-follower consensus control,” Mathematical Problems in Engineering, vol. 2016, Article ID 1048697, 8 pages, 2016.
View at: Publisher Site | Google Scholar | MathSciNet
S. Ghosh and J.-W. Lee, “Optimal finite-time consensus on fixed and periodic graphs,” SIAM Journal on Control and Optimization, vol. 51, no. 5, pp. 3946–3970, 2013.
View at: Publisher Site | Google Scholar | MathSciNet
R. Tedrake, Underactuated Robotics: Learning, Planning, and Control for Efficient and Agile Machines: Course Notes for MIT 6.832, Working draft edition, 2009.
J. Zhu, “A feedback optimal control by Hamilton-Jacobi-Bellman equation,” European Journal of Control, vol. 37, pp. 70–74, 2017.
View at: Publisher Site | Google Scholar | MathSciNet
Q. Wei, D. Liu, and H. Lin, “Value Iteration Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems,” IEEE Transactions on Cybernetics, vol. 46, no. 3, pp. 840–853, 2016.
View at: Publisher Site | Google Scholar
J. Qin, M. Li, Y. Shi, Q. Ma, and W. X. Zheng, “Optimal synchronization control of multiagent systems with input saturation via off-policy reinforcement learning,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–12, 2018.
View at: Publisher Site | Google Scholar
E. Semsar-Kazerooni and K. Khorasani, “Optimal consensus algorithms for cooperative team of agents subject to partial information,” Automatica, vol. 44, no. 11, pp. 2766–2777, 2008.
View at: Publisher Site | Google Scholar | MathSciNet
J. Wang, T. Yang, G. Staskevich, and B. Abbe, “Approximately adaptive neural cooperative control for nonlinear multiagent systems with performance guarantee,” International Journal of Systems Science, vol. 48, no. 5, pp. 909–920, 2017.
View at: Publisher Site | Google Scholar | MathSciNet
H. M. Guzey, H. Xu, and J. Sarangapani, “Neural network-based finite horizon optimal adaptive consensus control of mobile robot formations,” Optimal Control Applications and Methods, vol. 37, no. 5, pp. 1014–1034, 2016.
View at: Publisher Site | Google Scholar | MathSciNet
W. He and J. Cao, “Consensus control for high-order multi-agent systems,” IET Control Theory & Applications, vol. 5, no. 1, pp. 231–238, 2011.
View at: Publisher Site | Google Scholar | MathSciNet
G. Wen, Z. Duan, G. Chen, and W. Yu, “Consensus tracking of multi-agent systems with Lipschitz-type node dynamics and switching topologies,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 61, no. 2, pp. 499–511, 2014.
View at: Publisher Site | Google Scholar | MathSciNet
Y. Feng, Z. Duan, W. Ren, and G. Chen, “Consensus of multi-agent systems with fixed inner connections,” International Journal of Robust and Nonlinear Control, vol. 28, no. 1, pp. 154–173, 2018.
View at: Publisher Site | Google Scholar | MathSciNet
T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory (Classics in Applied Mathematics), Society for Industrial and Applied Mathematics, 2nd edition, 1999.
View at: MathSciNet
J. Engwerda, LQ Dynamic Optimization and Differential Games, John Wiley & Sons, 2005.
G. A. De Castro and F. Paganini, “Convex synthesis of controllers for consensus,” in Proceedings of the 2004 American Control Conference (AAC), vol. 6, pp. 4933–4938, Boston, MA, USA, July 2004.
View at: Google Scholar
P. Wang, Y. Li, Y.-H. Wang, and Z.-Q. Zhu, “A new method based on TOPSIS and response surface method for MCDM problems with interval numbers,” Mathematical Problems in Engineering, vol. 2015, Article ID 938535, 11 pages, 2015.
View at: Google Scholar
B. D. Anderson and J. B. Moore, Optimal Control: Linear Quadratic Methods, Prentice Hall, Englewood Cliffs, NJ, USA, 1990.
View at: MathSciNet
W. E. Boyce and R. C. DiPrima, Differential Equations Elementary and Boundary Value Problems, John Wiley & Sons, New York, NY. USA, 1997.
View at: MathSciNet
L. Vigano, M. Bergamasco, M. Lovera, and A. Varga, “Optimal periodic output feedback control: a continuous-time approach and a case study,” International Journal of Control, vol. 83, no. 5, pp. 897–914, 2010.
View at: Publisher Site | Google Scholar | MathSciNet

Copyright

Copyright © 2018 Mehrasa Ayatollahi et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

987

Downloads

762

Citations