Abstract and Applied Analysis

Volume 2014 (2014), Article ID 436362, 8 pages

http://dx.doi.org/10.1155/2014/436362

## On the Hamilton-Jacobi-Bellman Equation by the Homotopy Perturbation Method

^{1}Institute for Groundwater Studies, Faculty of Natural and Agricultural Sciences, University of the Free State, Bloemfontein 9300, South Africa^{2}Department of Mathematics, Texas A&M University, MSC 172, 700 University Boulevard, Kingsville, TX, USA^{3}Department of Mathematical Sciences, North-West University, Mafikeng Campus, Mmabatho 2735, South Africa

Received 16 April 2014; Revised 24 May 2014; Accepted 1 June 2014; Published 22 June 2014

Academic Editor: Guo-Cheng Wu

Copyright © 2014 Abdon Atangana et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Our concern in this paper is to use the homotopy decomposition method to solve the Hamilton-Jacobi-Bellman equation (HJB). The approach is obviously extremely well organized and is an influential procedure in obtaining the solutions of the equations. We portrayed particular compensations that this technique has over the prevailing approaches. We presented that the complexity of the homotopy decomposition method is of order . Furthermore, three explanatory examples established good outcomes and comparisons with exact solution.

#### 1. Introduction

Theory and application of optimal control have been widely used in different fields such as biomedicine, aircraft systems, and robotics. However, optimal control of nonlinear systems is a challenging task which has been studied extensively for decades. In the past two decades, the indirect methods have been extensively developed. It is well known that the nonlinear optimal control leads to a nonlinear two-point boundary value problem or a Hamilton-Jacobi-Bellman (HJB) partial differential equation. Many recent researches have been devoted to solving these two problems. In general, the HJB equation is a nonlinear partial differential equation that is hard to solve in most cases. An excellent literature review on the methods for solving the HJB equation is provided in [1], where a successive Galerkin approximation method is also considered. In the Galerkin approximation method a sequence of generalized Hamilton-Jacobi-Bellman equations is solved iteratively to obtain a sequence of approximations approaching the solution of Hamilton-Jacobi-Bellman equation. However, the proposed sequence may converge very slowly or even diverge. There are various efficient methods such as those reported in [2, 3] for the computation of open-loop optimal controls. However, feedback controls are much preferred in many engineering applications. Many other numerical methods [4–6]. In those, a sequence of nonhomogeneous linear time-varying TPBVPs is solved instead of directly solving the nonlinear TPBVP derived from the maximum principle. However, solving time-varying equations is much more difficult than solving time-invariant ones. Thus, it is required to solve HJB equation by an approximate-analytic method. Some of these equations were also evaluated in [7]; they used the so-called piecewise homotopy perturbation method to derive approximate solution for (HJB). Their method was a simple modification of the well-known homotopy perturbation method; the modification was based upon an algorithm in a sequence of small interval meaning time step for finding more accurate approximate solutions to the corresponding (HJB) equation.

In this paper we use homotopy decomposition method that was recently proposed in [8] to solve the Hamilton-Jacobi-Bellman equation. The method was first used to solve the groundwater flow equation [9]. To show the efficiency of the method, the following three problems are solved.

*Problem 1. *Consider a single-input scalar system as follows [10]:
The corresponding Hamiltonian function will be

*Problem 2. *Consider the following purely mathematical optimal control problem:
The corresponding Hamiltonian function will be

*Problem 3. *Consider the following nonlinear optimal control problem [11]:
The corresponding Hamiltonian function will be

The paper is organized as follows. In Section 2, the basics idea of homotopy decomposition method is presented. In Section 3, the advantages of the chosen method are presented. In Section 4, nonlinear time-variant HJB equation is presented. Conclusion is made in Section 5.

#### 2. Homotopy Decomposition Method [12, 13]

To exemplify the primitive thought of this approach we study an overall nonlinear nonhomogeneous partial differential equation with initial conditions of the following form: subject to the initial condition Somewhere, is a known function, is the general nonlinear differential operator, and represents a linear differential operator. The first step of the method here is to apply the inverse operator of both sides of (7) to obtain The multi-integral in (9) can be distorted to so that (9) can be reformulated as Employing the homotopy arrangement the resolution of the overhead integral equation is assumed in series form as and the nonlinear term can be decomposed as where is an embedding parameter. is He’s polynomials [14] that can be generated by The homotopy decomposition method is obtained by the beautiful coupling of decomposition method with He’s polynomials and is given by with Comparing the terms of same powers of , give solutions of various orders. The initial guess of the approximation is [13]. It is important to point out that the initial guess is Taylor series of order of the exact solution of the main problem [13].

##### 2.1. Advantage of the Method

The homotopy decomposition method is chosen to solve this nonlinear problem because of the following advantages the method has over the existing methods.(1)The method does not require the linearization or assumptions of weak nonlinearity.(2)The solutions are not generated in the form of general solution as in Adomian decomposition method. With ADM the recursive formula allows repetition of terms in the case of nonhomogeneous partial differential equation and this leads to the noisy solution [15].(3)The solution obtained is noise-free compared to the variational iteration method [16].(4)No correctional function is required as in the case of the variational homotopy decomposition method.(5)No Lagrange multiplier is required in the case of the variational iteration method [16].(6)It is more realistic compared to the method of simplifying the physical problems.(7)If the exact solution of the partial differential equation exists, the approximated solution via the method converges to the exact solution [8].(8)A construction of a homotopy is not needed as in the case of the homotopy perturbation method [14].

#### 3. Complexity of the Homotopy Decomposition Method

It is very important to test the computational complexity of the method or algorithm. Complexity of an algorithm is the study of how long a program will take to run, depending on the size of its input and length of loops made inside the code [13]. We compute a numerical example which is solved by the homotopy decomposition method. The code has been presented with Mathematica 8 according to the following code [13].

*Step 1. *Set .

*Step 2. *Calculating the recursive relation after the comparison of the terms of the same power is done.

*Step 3. *If with being the ratio of the neighbourhood of the exact solution [8] then go to Step 4; else and go to Step 2.

*Step 4. *Print out
as the approximation of the exact solution.

Lemma 1. *If the exact solution of the partial differential equation (7) exists, then
*

*Proof. *Let ; since the exact solution exists, then we have the following:

The last inequality follows from [13].

*Lemma 2. The complexity of the homotopy decomposition method is of order .*

*Proof. *The number of computations including product, addition, subtraction, and division is in Step 2: : 0 because, obtains directly from the initial guess [13], : 3, : 3.

Now in Step 4 the total number of computations is equal to .

*4. Nonlinear Time-Variant Hamilton-Jacobi-Bellman Equation*

*In this paper, we consider a general nonlinear control system described by
where a state is and vector is a control signal. The objective is to find the optimal control law , which minimizes the following cost function:
In this cost function, and are arbitrary convex functions and is final time of system operation. It is supposed [4] that
*

*Problem 4. *Consider a single-input scalar system as follows [9]:
The corresponding Hamiltonian function will be
Our concern here is to find , that is, stationary point for the Hamiltonian function. Therefore, differentiating (24) with respect to , we obtain
Applying the second derivative test, we obtain , since the second derivative is positive for all ; it follows that our turning point is a minimum, which is acceptable because our concern is to find the minimum value. Thus, by substituting in Hamilton-Jacobi-Bellman equation we obtain
To solve the above system of equation via the homotopy decomposition method, we transform it to the integral equation as follows:
Using the homotopy scheme the solution of the above integral equation is given in series form as
verifying
Comparing the terms of same powers of , we obtain the following system of integral equations:

*Lemma 3. If over its domain, then .*

*Proof. *Notice that
Replacing (32) in (26) we obtain
Arranging this we obtain
Solving the above, we obtain
Solving the integral equations we obtain the following terms:
Using the package Mathematica, in the same manner, one can obtain the rest of the components. But, here, 16 terms were computed and the asymptotic solution is given by
This verifies Lemma 3.

The analytical solution for this problem of state and the control is given as [4]
with
where, for simplicity,
To access the accuracy of the homotopy decomposition method, we determine numerically the absolute errors for as shown in Figure 1 that is showing the comparison between the two.

*Problem 5. *Consider the following purely mathematical optimal control problem:

The corresponding Hamiltonian function will be
Our concern here is to find , that is, stationary point for the Hamiltonian function. Therefore, differentiating (42) with respect to we obtain
Applying the second derivative test we obtain , since the second derivative is positive for all ; it follows that our turning point is a minimum, which is acceptable because our aim is to minimise. Thus, by substituting in Hamilton-Jacobi-Bellman equation we obtain
subjected to the following condition:
The exact solution of this form was proposed in [4] as
Shadowing the homotopy decomposition structure and equaling the relations of the same powers of , we obtain the following system of integral equations:
The following solutions are obtained. Here we chose the first term to be so that
With the similar routine one can acquire the remaining terms of the components. But 8 terms were computed and the asymptotic solution is given by
To access the accuracy of the homotopy decomposition method, we determine numerically the absolute errors for as shown in Figure 4. Figures 2 and 3 show approximate solution of equation and numerical error respectively.

*Problem 6. *Consider the following nonlinear optimal control problem [14]:
The corresponding Hamiltonian function will be
Following the discussion presented earlier, we obtain
Applying the second derivative test we obtain , since the second derivative is positive for all ; it follows that our turning point is a minimum, which is acceptable because our aim is to minimise. Thus, by substituting in Hamilton-Jacobi-Bellman equation we obtain
subjected to the following condition:
Following the homotopy decomposition scheme and comparing the terms of the same powers of , we obtain the following system of integral equations:
The following solutions are obtained. Here we chose the first term to be so that
The asymptotic solution is given by
Figure 5 shows the graphical representation of .

*5. Conclusion*

*The purpose of this paper was to use the homotopy decomposition method to solve the Hamilton-Jacobi-Bellman equation. The method is clearly a very efficient and powerful technique in finding the solutions of the equations. We presented some advantages the method has over the existing methods. We showed that the complexity of the homotopy decomposition method is of order . In addition, three illustrative examples demonstrated good results and comparisons with exact solution. Although the technique used in this paper is very accurate, we have to mention that there exist more appropriate techniques that can be used to handle this type of nonlinear equation.*

*Conflict of Interests*

*All authors confirmed that there is no conflict of interests in this paper.*

*Authors’ Contribution*

*Abdon Atangana wrote the first draft; Aden Ahmed and Suares Clovis Oukouomi Noutchie revised and corrected all mathematical and language mistakes; all three authors read and agreed to submit the final version of the paper.*

*References*

- R. W. Beard, G. N. Saridis, and J. T. Wen, “Approximate solutions to the time-invariant Hamilton-Jacobi-Bellman equation,”
*Journal of Optimization Theory and Applications*, vol. 96, no. 3, pp. 589–626, 1998. View at Google Scholar · View at Scopus - B. D. Craven,
*Control and Optimization*, Chapman & Hall, 1995. - K. L. Teo, C. J. Goh, and K. H. Wong,
*A Unified Computational Approach to Optimal Control Problems*, Longman Scientific & Technical, Essex, UK, 1991. - G. C. Wu and D. Baleanu, “Chaos synchronization of the discrete fractional logistic map,”
*Signal Processing*, vol. 102, pp. 96–99, 2014. View at Google Scholar - A. H. Bhrawy and M. A. Alghamdi, “A shifted Jacobi-Gauss-Lobatto collocation method for solving nonlinear fractional Langevin equation involving two fractional orders in different intervals,”
*Boundary Value Problems*, vol. 2012, article 62, 2012. View at Publisher · View at Google Scholar · View at Scopus - G. C. Wu and D. Baleanu, “Discrete chaos in fractional delayed logistic maps,”
*Nonlinear Dynamics*, 2014. View at Publisher · View at Google Scholar - S. Effati, H. S. Nik, and M. Shirazian, “An improvement to the homotopy perturbation method for solvingthe Hamilton-Jacobi-Bellman equation,”
*IMA Journal of Mathematical Control and Information*, 2013. View at Google Scholar - A. Atangana and A. Secer, “The time-fractional coupled-Korteweg-de-Vries equations,”
*Abstract and Applied Analysis*, vol. 2013, Article ID 947986, 8 pages, 2013. View at Publisher · View at Google Scholar · View at Scopus - S. A. Yousefi, M. Dehghan, and A. Lotfi, “Finding the optimal control of linear systems via He's variational iteration method,”
*International Journal of Computer Mathematics*, vol. 87, no. 5, pp. 1042–1050, 2010. View at Publisher · View at Google Scholar · View at Scopus - A. Jajarmi, N. Pariz, A. V. Kamyad, and S. Effati, “A novel modal series representation approach to solve a class of nonlinear optimal control problems,”
*International Journal of Innovative Computing, Information and Control*, vol. 7, no. 3, pp. 1413–1425, 2011. View at Google Scholar · View at Scopus - J.-H. He, “Homotopy perturbation technique,”
*Computer Methods in Applied Mechanics and Engineering*, vol. 178, no. 3-4, pp. 257–262, 1999. View at Google Scholar · View at Scopus - A. Atangana and J. F. Botha, “Analytical solution of the groundwater flow equation obtained
*via*homotopy decomposition method,”*Journal of Earth Science & Climatic Change*, vol. 3, article 115. View at Publisher · View at Google Scholar - A. Atangana and D. Baleanu, “Nonlinear fractional Jaulent-Miodek and Whitham-Broer-Kaup equations within Sumudu transform,”
*Abstract and Applied Analysis*, vol. 2013, Article ID 160681, 8 pages, 2013. View at Publisher · View at Google Scholar · View at Scopus - A. Atangana and A. Kılıçman, “Analytical solutions of boundary values problem of 2D and 3D poisson and biharmonic equations by homotopy decomposition method,”
*Abstract and Applied Analysis*, vol. 2013, Article ID 380484, 9 pages, 2013. View at Publisher · View at Google Scholar - A. Atangana, “New class of boundary value problems,”
*Information Sciences Letters*, no. 12, pp. 67–76, 2012. View at Google Scholar - G. C. Wu and D. Baleanu, “Discrete fractional logistic map and its chaos,”
*Nonlinear Dynamics*, vol. 75, pp. 283–287, 2014. View at Google Scholar

*
*