Mathematical Problems in Engineering

Volume 2013, Article ID 713725, 9 pages

http://dx.doi.org/10.1155/2013/713725

## Precise Integration Method for Solving Noncooperative LQ Differential Game

^{1}Department of Engineering Mechanics, State Key Laboratory of Structural Analysis of Industrial Equipment, Dalian University of Technology, Dalian 116024, China^{2}School of Aeronautics and Astronautics, State Key Laboratory of Structural Analysis of Industrial Equipment, Dalian University of Technology, Dalian 116024, China

Received 18 March 2013; Accepted 23 May 2013

Academic Editor: Bozidar Sarler

Copyright © 2013 Hai-Jun Peng et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

The key of solving the noncooperative linear quadratic (LQ) differential game is to solve the coupled matrix Riccati differential equation. The precise integration method based on the adaptive choosing of the two parameters is expanded from the traditional symmetric Riccati differential equation to the coupled asymmetric Riccati differential equation in this paper. The proposed expanded precise integration method can overcome the difficulty of the singularity point and the ill-conditioned matrix in the solving of coupled asymmetric Riccati differential equation. The numerical examples show that the expanded precise integration method gives more stable and accurate numerical results than the “direct integration method” and the “linear transformation method”.

#### 1. Introduction

The research for the differential game is derived from the military field, such as the pursuit evasion problem, the spacecraft interception problem, and the cooperative or noncooperative problem. The emergence of the Isaacs’s monograph [1] gives birth to the differential games. Particularly, the differential game has been introduced into the economic field by Nash [2], and the equilibrium solution of the LQ differential game had been named by Nash-equilibrium solution. So far, many works for the LQ differential games have been done by researchers.

The qualitative analysis about the LQ differential game, that is, the existence, stability of solutions, has been reported in many references. Reference [3] gives the series Nash solution of two-person nonzero-sum LQ differential game. Reference [4] discusses the Nash-equilibrium point and the existence, uniqueness of the algebraic Riccati equation for the closed LQ differential game. Reference [5] proposes the global existence of solutions to coupled Riccati differential equation (CRDE) in closed-loop Nash games. In contrast to the qualitative analysis, the numerical method for solving the LQ differential game problem to obtain the high performance solutions is still a challenging problem. Reference [6] founds that a special type of LQ differential game problem has the character of Hamiltonian system and applies the symplectic Runge-Kutta algorithm for solving this differential game problem. Reference [7] employs the Magnus integrators for solving -player LQ differential games, and the CRDE which is reformulated as a linear ordinary equation is solved by Magnus integrators.

From the above discussions, the key for solving LQ differential game is to solve the CRDE. Only a few problems can obtain their analytical solutions. It is necessary to employ numerical methods for solving the CRDE. The traditional numerical method is to employ an ordinary differential equation (ODE) solver for backward integration of the nonlinear CRDE. This kind of numerical methods is effective in most of cases. However, it may fail when the singularity point is encountered. Besides, the CRDE is also usually reformulated as a linear ODE such as [6, 7]. The advantage of this transformation method is that the linear ODE can be easier solved by comparing with the nonlinear ODE. However, the inverse matrix exists for the expression of the CRDE. Therefore, it restricts the application of this numerical method for many LQ differential game problems.

In the field of LQ optimal control, the uncoupled Riccati differential equation (URDE) has been successfully solved by the precise integration method (PIM) [8–10]. Particularly, the PIM can obtain the machine accuracy solutions even for stiff problems. However, the difference between URDE and CRDE is obvious, and the CRDE is much difficult to be solved. There is no PIM for solving this CRDE from the LQ differential game problem. Therefore, it is partly inspired by the high accuracy of the PIM for solving the URDE, and we employ the adaptive parameter criterion to expand the PIM and apply it for solving the CRDE in this paper. Finally, the noncooperative LQ differential game problem can be solved by the proposed expanded PIM.

#### 2. Noncooperative LQ Differential Game

##### 2.1. Problem Description

For the general case with -players, the controlled equation can be defined for the LQ differential game problem as follows: The quadratic cost function associated with the player is given by where denotes the fixed prescribed duration of the game, is the initial state, , and . The weighting matrices , , and are assumed to be nonnegative; is assumed to be positive definite.

*Definition 1. * The noncooperative -players LQ differential game is that the purpose of each player is to minimize the cost function in (2) under the controlled differential equation (1). It means that

If each player employs strategy (3), then a conflict situation will be provoked. Thus a definition of an equilibrium behavior which in some sense is suitable for all players is demanded.

*Definition 2. *The system is described by (1), (2), and (3); a strategy is said to be Nash-equilibrium strategy if for any admissible :

This definition means that if all players are applying Nash-equilibrium strategies , then there is no sense to change them unilaterally; that is, a player may only increase its losses.

##### 2.2. Coupled Matrix Riccati Differential Equation

From the above description, the standard solution for a noncooperative and nonzero-sum LQ differential game problem can be obtained by calculus of variations. The theorem can be extracted from [11] as follows.

Theorem 3. * For the -players differential game, there exists a unique solution set to the coupled matrix Riccati differential equations:
**
where . Then, the LQ differential game admits a unique Nash-equilibrium solution given by
**
The proof can be found at [11].*

From the above theorem, the key for obtaining the Nash-equilibrium strategy of LQ differential game is to find the solution of the coupled matrix Riccati differential equation (5). However, it is not an easy task to find the solution of (5). It is comparatively rare that an exact analysis can be made. Numerical methods must be employed to solve this coupled matrix Riccati differential equation in most cases.

The traditional or the most obvious method is to employ a numerical integration method. Then, the direct integration from the end time to the initial time 0 can be implemented by using an ordinary differential solver, such as ODE45 function in MATLAB software. However, this direct integration method has some difficulty in practical implementation. Because the coupled Riccati differential equations are high nonlinear differential equations, another method is to transfer the coupled Riccati differential equations into the linear ODE. Then it is solved by an ordinary differential solver.

If we take all the into a single whole matrix, the coefficient matrices of (5) can be rewritten and composed as follows: Then the coupled matrix Riccati differential equation (5) can be rewritten as

At last, the coupled matrix Riccati differential equations are transferred into a single whole asymmetric Riccati differential equation.

Theorem 4. * The solution of (10) can be obtained by solving a linear ODE. Let be the solution of the following linear ODE with the terminal boundary condition:
**
By solving linear ODE (11), the solution of nonlinear asymmetric Riccati differential equation can be obtained by
**
The proof can be found at [12].*

From the above process, we can see that the solving of linear ODE is easier than the solving of nonlinear ODE. However, if the matrices or are ill-conditioned, then the solution of matrix Riccati differential equation (5) or asymmetric Riccati differential equation (10) cannot be obtained easily.

#### 3. Improved Precise Integration Method

In this section, we will firstly give the fundamental PIM for solving the asymmetric Riccati differential equation. In order to increase the efficiency of the fundamental PIM, then an improved precise integration method which is proposed by [10] for solving the symmetric Riccati differential equation will be expanded for solving the asymmetric Riccati differential equation.

##### 3.1. PIM for Solving Asymmetric Riccati Differential Equation

Zhong and Zhu [8] firstly proposed the PIM for solving the symmetric Riccati differential equation (SRDE) by using the relationship between the solutions of SRDE and the fundamental solutions of the linear Hamiltonian system. Then this idea is developed for solving the asymmetric Riccati differential equation (ARDE) [9]. In this section, a brief introduction of PIM for solving ARDE is given.

The PIM connects the coefficient matrices of the ARDE with the linear ODE as follows:

To find the solutions for the general ARDE, the relationship between the state vectors at time and the state vectors at time is introduced. If the time interval is considered as a subinterval within the whole integration interval , this relationship can be formulated as

It can be proved that the interval matrices , and satisfy the following differential equations and boundary conditions [9]: where and are and dimension identity matrix, respectively.

Comparing (15) with (10), it shows that both the matrix and the matrix satisfy the same differential equation. Therefore, if the matrix defined by (14) can be obtained, then the solutions of the ARDE (10) can be given. In order to solve the ARDE, the continuous time should be discreted into a series of time intervals; that is, . According to the PIM, the subinterval is further divided into extremely small intervals with the equal length . The matrices , and can be expressed as the function of the time interval by using a fourth-order Taylor series, and then the matrices and can be combined times with the following formulae:

Then the matrices can be obtained by using the following formulae:

Because the matrices , and for the subinterval have been computed, the matrices , and at time can be obtained by implementing the following formulae:

According to the theorem of the computational structural mechanics [13], when the length of subinterval is zero, the subinterval matrices , will become zero matrices. When the length of subinterval is going to infinity, the subinterval matrices , will become constant matrices. For the linear system, the inversion of the matrices and can always be guaranteed [13]. Furthermore, because the matrix and the matrix satisfy the same differential equation, the solution for the ARDE (10) can be obtained according to (19):

Thus, the PIM for solving the ARDE has been given by implementing the formulae (20) at last. From the above procedure of the PIM, the key of the PIM is the computation of the interval matrices , and . The original PIM employs the Taylor series method with the fixed number of intervals . With the increasing , the precision of the PIM will increase. However, the computational work will increase much, especially for a large degree of freedom problem. Therefore, a main issue is how to compute the interval matrices , and with a higher precision and a lower computational work.

##### 3.2. Improved Parameter Computation for PIM

Since the accuracy and efficiency of the PIM are highly dependent on the choices of parameter , [10] has proposed an improved precise integration method (IPIM) for solving the symmetric Riccati differential equation. Therefore, in this paper, we expand the idea of the IPIM and propose the adaptive PIM for solving the asymmetric Riccati differential equation.

The relationship between the adjacent state vectors and the state vectors in the time interval has been given in (14) by employing the interval matrices , and . Furthermore, this relationship can also be expressed with the solution of linear ODE (11) or (13) as follows: where denotes the exponential of the matrix . Comparing (14) with (21), we can get the following relationship:

In fact, on the one hand, the length of interval cannot be made arbitrarily large. This is because the columns of become more linearly dependent on the length of intervals [14]. Consequently, if the choice of length of interval is not infinite and appropriate, the inversion of the matrix always exists. On the other hand, the computing of the exponential of the matrix and then transforming it into interval matrices , and using (22) are feasible and stable for a small interval.

The implementation of the PIM for computing the matrix exponential in one time step can be given as follows: Since the time step is not very large and is extremely small, then the Taylor series approximation method can be used to evaluate ; that is, where Substituting (24) into (23) gives Meanwhile, the addition theorem is applied only to the incremental part , and it gives Finally, by performing (27) times, the matrix exponential is obtained by

In order to choose the appropriate parameters and , a theorem has been given by [15] for the scaling and squaring method. Actually, the PIM considers the roundoff errors in the computational procedure. However, the scaling and squaring method does not. Therefore, the PIM is more stable. In this paper, we employ Theorem 5 to choose the parameters and adaptively.

Theorem 5. * If , then , where
**
in which denotes the norm of the matrix and denotes the Taylor series of order .*

According to the analysis given above, the detailed implementation of the adaptive PIM for the ARDE is listed as follows.

*Algorithm 6. * Form the coefficient matrices , , and of the ADRE by (7)~(10), then the assembled matrix can be obtained by (11);

Give the duration of the game time , then the time step is given by ;

Let and then determine the two parameters and according to the adaptive error Theorem 5 for a given error tolerance;

Compute the -order of the Taylor series approximation formulation by (25);

Perform the following steps For ; End

Get the matrix exponential by (28) and then transform it into the interval matrices by (22);

Compute interval matrices at time by (19), and then the solution of the ARDE (10) can be obtained from the recursive formula (20);

The Nash equilibrium solution of the LQ differential game can be given by (6).

#### 4. Numerical Examples

In this section, two examples are given to show the validity of the PIM for solving noncooperative LQ differential game problem. Meanwhile, two other numerical methods are also employed to be compared with the PIM. The first numerical method is to directly integrate the coupled matrix Riccati differential equation (5) by ODE45 solver in MATLAB software. We call this first method “direct integration method”. The second numerical method is to integrate the linear ODE (11) by ODE45 solver in MATLAB software. We call this second method “linear transformation method”. Because an adaptive step Runge-Kutta is implemented in ODE45 solver, the relative error and the absolute error are both set as for the “direct integration method” and the “linear transformation method”.

*Example 7. * Considering a simple pursuit-evasion problem in space [7], the symbol denotes the control acceleration of the purser, and the symbol denotes the control acceleration of the evader. The symbol denotes the relative position of the purser with respect to the evader. The differential equation can be expressed by
If we denote the state vector , then (30) can be rewritten as the following:
The cost functions are selected by
The exact solutions of the coupled matrix Riccati differential equation are

From the exact solutions (33), it is shown that the parameter has important influence on the coupled matrix Riccati differential equation. Hence, two cases are considered, that is, Case 1“” and Case 2 “”. In both cases, the different numerical results are denoted by different symbols. The symbol “” denotes the solution from the PIM, the symbol “×” denotes the solution from the “direct integration method”, and the symbol “” denotes the solution from the “linear transformation method”.

*Case 1 (). *The length of step for the PIM is given as . Figure 1 shows the numerical solutions of the coupled matrix Riccati differential equation obtained from the PIM, the “direct integration method” and the “linear transformation method” for Case 1. It can be easily seen that the symbols “”, “×”, and “” overlap each other, and they are all nearly on the line of the exact solutions. Therefore, the above three methods all have enough precisions for Case 1.

*Case 2 (). *The length of step for the PIM is also given as . Figure 2 shows the numerical solutions of the coupled matrix Riccati differential equation obtained from the above three methods for Case 2. The three methods overlap each other after about . Before , the PIM and the “linear transformation method” are both nearly on the line of the exact solutions. However, the results from the “direct integration method” disappear in Figure 2. Therefore, the “direct integration method” fails for Case 2.

From the exact solutions (33), it can be seen that the function will change much for different parameter . We plot the curves of the function for Cases 1 and 2 in Figure 3. The line corresponding to Case 1 has no zero point. However, the line corresponding to Case 2 has a zero point. Consequently, we find the reason for the failure of the “direct integration method” in Case 2. Because the “direct integration method” which integrates the coupled matrix Riccati different equation from the end time to the initial time 0 cannot go through the singularity point, the integration procedure stops at this singularity point.

*Example 8. * Considering another LQ differential game problem [5], the controlled differential equation can be expressed by
The weighting matrices in the cost function are given by
The fixed prescribed duration of the game is .

This example has no analytical solutions, and we also employ three different numerical methods for solving this example. In Figure 4, the solid lines denote the solution from the PIM, and the time step length is ; the symbol “×” denotes the solution from the “direct integration method”, and the symbol “” denotes the solution from the “linear transformation method”.

From Figure 4, we can see that the numerical results from the “direct integration method” are nearly the same with the numerical results from the PIM in the all range of time . However, the numerical results from the “linear transformation method” have much difference with the above two methods. Particularly, in the range of time interval , the “linear transformation method” gives the wrong results. Therefore, the “linear transformation method” fails for this example.

The key point of the “linear transformation method” can be easily found in (12) in Theorem 4. Because the inversion matrix of exists in (12), if the matrices is ill-conditioned, then the solution of the coupled matrix Riccati differential equation (5) cannot be obtained. In order to find the failure reason of the “linear transformation method” for this example, we have given the determinants of the matrix functions and in Figure 5. We can see that the determinants of the matrix functions and from time to time increase much linearly. Therefore, the matrices functions and are both ill-conditioned matrices, and these causes led to the bad results.

#### 5. Conclusions

In order to obtain the Nash-equilibrium strategy of the -players noncooperative LQ differential game problem, the key is to solve the coupled matrix Riccati differential equation. The precise integration method is expanded and applied for solving the coupled matrix Riccati differential equation in this paper. Furthermore, a comparison of the algorithm performance between the precise integration method, the “direct integration method”, and the “linear transformation method” is also given. At last, the numerical examples show that the “direct integration method” cannot go through the singularity point, and the integration procedure stops at this singularity point. The “linear transformation method” fails in the context of the ill-conditioned or singular matrix. However, the expanded precise integration method can always work and overcome the above difficulties. Therefore, the expanded precise integration method gives the more stable and accurate numerical results.

#### Acknowledgments

The project is supported by the National Science Foundation of China (11102031, 11072044) and the Fundamental Research Funds for the Central Universities (DUT13LK25).

#### References

- R. Isaacs,
*Differential Games. A Mathematical Theory with Applications to Warfare and Pursuit, Control and Optimization*, John Wiley & Sons, New York, NY, USA, 1965. View at Zentralblatt MATH · View at MathSciNet - J. Nash, “Non-cooperative games,”
*Annals of Mathematics*, vol. 54, pp. 286–295, 1951. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - J. B. Cruz Jr. and C. I. Chen, “Series Nash solution of two-person, nonzero-sum, linear-quadratic differential games,”
*Journal of Optimization Theory and Applications*, vol. 7, pp. 240–257, 1971. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - G. P. Papavassilopoulos and J. B. Cruz Jr., “On the uniqueness of Nash strategies for a class of analytic differential games,”
*Journal of Optimization Theory and Applications*, vol. 27, no. 2, pp. 309–314, 1979. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - G. Freiling, G. Jank, and H. Abou-Kandil, “On global existence of solutions to coupled matrix Riccati equations in closed-loop Nash games,”
*Institute of Electrical and Electronics Engineers*, vol. 41, no. 2, pp. 264–269, 1996. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - Z. X. Xu, D. Y. Zhou, and Z. C. Deng, “Numerical method based on Hamilton system and symplectic algorithm to differential games,”
*Applied Mathematics and Mechanics*, vol. 27, no. 3, pp. 341–346, 2006. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - S. Blanes and E. Ponsoda, “Magnus integrators for solving linear-quadratic differential games,”
*Journal of Computational and Applied Mathematics*, vol. 236, no. 14, pp. 3394–3408, 2012. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - W. X. Zhong and J. P. Zhu, “Precise time integration for the matrix Riccati differential equation,”
*Journal on Numerical Methods and Computer Applications*, vol. 17, no. 1, pp. 26–35, 1996. View at Google Scholar · View at MathSciNet - W. X. Zhong, “Combined method for the solution of asymmetric Riccati differential equations,”
*Computer Methods in Applied Mechanics and Engineering*, vol. 191, no. 1-2, pp. 93–102, 2001. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - Q. Gao, S. J. Tan, W. X. Zhong, and H. W. Zhang, “Improved precise integration method for differential Riccati equation,”
*Applied Mathematics and Mechanics*, vol. 34, no. 1, pp. 1–14, 2013. View at Google Scholar - T. Başar and G. J. Olsder,
*Dynamic Noncooperative Game Theory*, vol. 160 of*Mathematics in Science and Engineering*, Academic Press, New York, NY, USA, 1982. View at MathSciNet - W. T. Reid,
*Riccati Differential Equations*, vol. 86, Academic Press, New York, NY, USA, 1972. View at MathSciNet - W. X. Zhong,
*Duality System in Applied Mechanics and Optimal Control*, Kluwer Academic, 2004. - E. J. Davison and M. C. Maki, “The numerical solution of the matrix Ricatti differential equation,”
*IEEE Transactions on Automatic Control*, vol. 18, no. 1, pp. 71–73, 1973. View at Google Scholar - C. Moler and C. Van Loan, “Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later,”
*SIAM Review*, vol. 45, no. 1, pp. 3–49, 2003. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet