Mathematical Problems in Engineering

Volume 2013 (2013), Article ID 914741, 10 pages

http://dx.doi.org/10.1155/2013/914741

## Application of Optimal HAM for Finding Feedback Control of Optimal Control Problems

^{1}Department of Mathematics, Neyshabur Branch, Islamic Azad University, Neyshabur, Iran^{2}Department of Mathematics, University of Venda, Private Bag X5050, Thohoyandou 0950, South Africa

Received 8 November 2012; Accepted 1 February 2013

Academic Editor: Trung Nguyen Thoi

Copyright © 2013 H. Saberi Nik and Stanford Shateyi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

An optimal homotopy-analysis approach is described for Hamilton-Jacobi-Bellman equation (HJB) arising in nonlinear optimal control problems. This optimal approach contains at most three convergence-control parameters and is computationally rather efficient. A kind of averaged residual error is defined. By minimizing the averaged residual error, the optimal convergence-control parameters can be obtained. This optimal approach has general meanings and can be used to get fast convergent series solutions of different types of equations with strong nonlinearity. The closed-loop optimal control is obtained using the Bellman dynamic programming. Numerical examples are considered aiming to demonstrate the validity and applicability of the proposed techniques and to compare with the existing results.

#### 1. Introduction

Many real-world problems in engineering, economics, and biomedicine can be formulated as optimal control problem of the form where is the control function, is the state function, constant parameters, and are the initial and the final states, is the salvage value or scrap value function, is the vector-valued transition function, is the integrand function, and is the dummy variable of integration rather than .

There are various efficient methods such as those reported in [1–3] for the computation of open-loop optimal controls. However, feedback controls are much preferred in many engineering applications. In order to determine the optimal control law, there is an approach using dynamic programming [4]. This approach leads to the HJB equation that is hard to solve in most cases. An excellent literature review on the methods for solving the HJB equation is provided in [5], where a successive Galerkin approximation (SGA) method is also considered. In the SGA, a sequence of generalized HJB equations is solved iteratively to obtain a sequence of approximations approaching the solution of HJB equation. However, the proposed sequence may converge very slowly or even diverge. Another approach is to treat the problem with a measure theory approach [6]. This changes the nonlinear OCP to a linear programming and gives a piecewise constant control law. In the past two decades, the indirect methods have been extensively developed. It is well known that the nonlinear OCP leads to a nonlinear two-point boundary value problem (TPBVP) or a Hamilton-Jacobi-Bellman partial differential equation. Many recent researches have been devoted to solve these two problems.

In particular, we will derive the fundamental first-order partial differential equation obeyed by the optimal value function, known as the HJB equation. This shift in our attention, moreover, will lead us to a different form for the optimal value of the control vector, namely, the feedback or closed-loop form of the control. This form of the optimal control typically gives the optimal value of the control vector as a function of the current date, the current state, and the parameters of the control problem. In contrast, the form of the optimal control vector derived via the necessary conditions of optimal control theory is termed open loop and in general gives the optimal value of the control vector as a function of the independent variable time, the parameters, and the initial and/or terminal values of the planning horizon and the state vector. Essentially, the feedback form of the optimal control is a decision rule, for it gives the optimal value of the control for any current period and any admissible state in the current period that may arise. In contrast, the open-loop form of the optimal control is a curve, for it gives the optimal values of the control as the independent variable time varies over the planning horizon. Thus, it is required to solve an HJB equation by numerical-analytic-approximate method. In [7], the authors used the basic ADM for an example of HJB equation and then finding optimal control signal. In [8–10], standard HPM, piecewise HPM and differential transform method (DTM) were used for finding feedback control of optimal control problems.

One of the semiexact methods for solving linear and nonlinear equation which does not need small/large parameters is homotopy analysis method (HAM), first proposed by Liao in 1992 [10–13]. Since Liao’s book [14] for the homotopy analysis method was published in 2003, more and more researchers have been successfully applying this method to various nonlinear problems in science and engineering, such as the viscous flows of non-Newtonian fluids [15], the KdV-type equations [16], and finance problems [17].

The HAM contains a certain auxiliary parameter , which provides us with a simple way to adjust and control the convergence region and rate of convergence of the series solution. Moreover, by means of the so-called -curve, it is easy to determine the valid regions of to gain a convergent series solution. Thus, through HAM, explicit analytic solutions of nonlinear problems are possible.

However, as illustrated in [18], the usual HAM has only one convergence-control parameter , and it is a pity that curves for convergence-control parameter (i.e., -curves) cannot tell us which value of gives the fastest convergent series. Recently, to overcome this shortcoming, Liao [18] proposed an optimal HAM with more than one convergence-control parameter. In this optimal method, Liao introduced the so-called averaged residual error to get the optimal convergence-control parameters efficiently. In general, the optimal HAM can greatly modify the convergence of homotopy-series solution.

The results obtained in this paper show that the solutions given by the optimal HAM give much better approximations and convergence much faster than those given by the usual HAM. In this paper, we employed a new proposal of Liao [18, 19], namely, optimal homotopy-analysis approach to solve an HJB equation obtained from (2)–(4). We use here special deformation functions, which are determined completely by only one characteristic parameter and , respectively. In this way, there exist at most only three convergence-control parameters , and at any order of approximations.

#### 2. Nonlinear Time-Variant HJB Equation

Consider a process described by the following system of nonlinear differential equations, which is called the equation of motion, on a fixed interval where is the state vector. Let be a closed set. A piecewise continuous function is said to be an admissible control if . Let be the class of such admissible controls. The function is a real-valued continuously differentiable function and has continuous first partial derivative with respect to . The initial condition for (2) is Along with this controlled process, we have a cost functional of the form Here, is the running cost, and is the terminal cost. This cost functional depends on the initial position and the choice of control . The optimization problem is therefore to minimize , for each , over all controls . The pair which achieves this minimum is called an optimal control. In fact, the optimization problem with performance index as in (4) is called a Bolza problem.

In dynamic programming, the minimum value of the performance index is considered as a function of this initial point. This function is called the value function. Whenever the value function is differentiable, it satisfies a nonlinear first-order hyperbolic partial differential equation called the partial differential equation of dynamic programming. This equation is used for constructing a nonlinear optimal feedback control law. If we consider a family of optimization problems with differential initial condition , we consider the dependence of the value of these optimization problems on their initial conditions. Thus, a value function is defined by

Theorem 1 .. *Let be any interior point of the set at which the function is differentiable. Function satisfies the partial differential inequality
**
for all . If there is an optimal control , then the partial differential equation
**
is satisfied [20].*

This nonlinear time-variant differential equation is called “HJB equation.” We have the following boundary condition: and by introducing the Hamiltonian function we have Therefore, by substitution of Hamiltonian function (10) in (7), we have

#### 3. Basic Idea of Optimal Homotopy Analysis Method

In this section, we give a brief description of optimal homotopy-analysis approaches in general cases. For a general nonlinear differential/integral equation as follows: where is an unknown function, and denote, respectively, spatial and temporal independent variables, we can always choose a proper initial guess and a proper auxiliary linear operator to construct the so-called zeroth-order deformation equation where is a deformation function with unknown convergence-control parameters denoted by , and is a deformation function with ) unknown convergence-control parameters denoted by , respectively. Here, Put . Then, we have unknown convergence-control parameters . Assuming that the initial guess , the auxiliary linear operator , and the convergence-control parameters are so properly chosen that the Taylor series converges at , we have the homotopy-series solution Substituting the series (15) into the zeroth-order deformation equation (13) and then equating the coefficients of the like power of the embedding parameter , we have the high-order deformation equation. Consider where and are coefficients of the Taylor series Let be complex functions analytic in the region , which satisfy The special solution of (17) is given by where and is the inverse operator of .

Now, we define a kind of averaged residual error [18] The value of is depends to problem, for Examples 2 and 3, we have , and , respectively.

Note that contains unknown convergence-control parameters , whose optimal values are given by the minimum of , corresponding to a set of nonlinear algebraic equations So, the above approach is called the -parameter optimal homotopy-analysis approach. In general, the above-mentioned optimal homotopy-analysis approaches can greatly modify the convergence of homotopy-series solution. And the optimal homotopy-analysis approaches with one or two unknown convergence-control parameters are strongly suggested; an optimal approach with too many unknown convergence-control parameters is not efficient computationally.

#### 4. Application

To illustrate the effectiveness of the optimal homotopy analysis method, we will consider two examples. Comparisons are made between the exact solution and optimal homotopy-analysis approaches.

*Example 2. *Consider the following purely mathematical optimal control problem:
Suppose that we consider the following cost function for this system:
The Hamiltonian function will be
Since the Hamiltonian, which as you know is the right-hand-side expression to be minimum, is a convex function of the state and control variables, a solution of the necessary conditions is a solution of the optimal control. Thus, the necessary and sufficient condition for the above minimization problem in the HJB equation is given by
which is easily solved to get
Because =>, is a minimum and acceptable. Now, by substitution of in HJB equation, we have the following:
In Chapter 19 of [21], the authors obtained the solution of the above HJB equation in the form
for simplicity, let , where
To solve (30) by means of optimal homotopy analysis method, we choose the initial approximation, final time, and auxiliary linear operator
There are an infinite number of deformation functions satisfying the properties (19). For the sake of computation efficiency, we use here the following one-parameter deformation functions:
where and are constants, called the convergence-control parameter, and
The different values of give different paths of , as shown in Figure 1.

Let denote the embedding parameter, an auxiliary parameter, called the convergence-control parameter, and a kind of continuous mapping of , respectively. Obviously, when and , it holds respectively. Thus, as increases from to , the solutions vary from the initial guesses to the solutions . We construct the zeroth-order deformation equation: subject to the boundary condition As mentioned, we have the th-order deformation equation where and the coefficients are defined by (35). Let denote a special solution of (39) and the inverse operator of , respectively. We have We now successively obtain

It should be emphasized that contains at most three unknown convergence-control parameters , and , which determine the convergence region and rate of the homotopy-series solution. Obviously, if the convergence-control parameters , and are properly chosen, the homotopy-series solution may converge fast. So, we should find out the good enough values of , and , so that the homotopy-series solution converges fast enough.

##### 4.1. Comparisons of Different Approaches for Example 2

In this section, we will give optimal homotopy-analysis approaches with different numbers of unknown convergence-control parameters and compare them in detail.

###### 4.1.1. Optimal in Case of

In this case, only one convergence-control parameter is unknown. For given order of approximation , the optimal value of is given by the minimum of , corresponding to a nonlinear algebraic equation .

The curves of versus at a different order of approximation , and are shown in Figure 2. In case of , there is only one unknown convergence-control parameter ; thus the optimal value of is determined by the minimum of , corresponding to the nonlinear algebraic equation . According to Table 1, the value minimum of is obtained at . It should be noted that, when we fixed and , automatically the problem turns into the so-called homotopy perturbation method (HPM). So, the HPM is only a special case of the HAM when .

In case of , one has the plain deformation functions , which is exactly the same as that used in traditional HAM.

###### 4.1.2. Optimal in Case of

In this case, has the value minimum of at the optimal point . This illustrates that the second one-parameter optimal homotopy-analysis approach is as good as the first one mentioned in Section 4.1.1. (See Table 2).

###### 4.1.3. Optimal in Case of

In this case, we obtain the unknown convergence-control parameters and in case of . The corresponding residual error is now a function of both and , which has the value minimum of at the optimal point , . (See Table 3).

###### 4.1.4. Optimal , and in Case of

Let us consider the optimal approach with the three unknown convergence-control parameters. The corresponding is now a function of , and , which has the minimum at the optimal values , , and . The corresponding homotopy-series solution converges much faster than homotopy-series solution in case of and , as shown in Table 4.

Based on the above calculations for this equation, we found that all optimal homotopy-analysis approaches can give much better approximations, which converge much faster than those without optimal convergence-control parameters in case of and , as shown in Tables 1–4.

The absolute error of the 20th-order approximate solution and analytic solution (31) with , , and , is shown in Figures 3 and 4, respectively. Comparison of the analytic solution (31) with the optimal homotopy analysis method solution by and is given in Figure 5. The absolute errors have been calculated for different cases of , and in Table 5.

*Example 3. *Consider a single-input scalar system as follows:
The corresponding Hamiltonian function will be
For finding , we have
Therefore, we obtain
Because , is a minimum and acceptable. Now, by substitution of in HJB equation, we have the following:
or
Note that, by (29), the optimal control law is , where the exact solution of is [2]
To solve (48) by means of optimal homotopy analysis method, we choose the initial approximation and auxiliary linear operator
We construct the zeroth-order deformation equation
subject to the boundary condition
As mentioned, we have the th-order deformation equation
where
and the coefficients are defined by (35). Let denote a special solution of (53) and the inverse operator of , respectively. We have
We now successively obtain

##### 4.2. Comparisons of Different Approaches for Example 3

In this section, we will give optimal homotopy-analysis approaches with different numbers of unknown convergence-control parameters and compare them in detail.

###### 4.2.1. Optimal in Case of

In this case, only one convergence-control parameter is unknown. We solved this example by standard HAM, and we investigated the influence of by means of the so-called -curves. As pointed out by Liao [14], the valid region of is a horizontal line segment. So, we can just determine the possible valid region of as shown in Figure 6. However, usually the -curves cannot tell us which value of gives the fastest convergent series.

To overcome this shortcoming, we determined the possible optimal value of by the minimum of averaged residual error , corresponding to the nonlinear algebraic equation . According to Table 6, the value minimum of is obtained at . The curves of versus at different order of approximation , and are shown in Figure 7.

###### 4.2.2. Optimal in Case of

In this case, has the value minimum of at the optimal point . Our calculations showed that the corresponding in case of , has the minimum . This illustrates that the second one-parameter optimal homotopy-analysis approach is not obviously better than the first one mentioned in Section 4.2.1. But this approach can give much better approximation than those without optimal convergence-control parameters in case of and .

###### 4.2.3. Optimal in Case of

In this case, we obtain the unknown convergence-control parameters and in case of . The corresponding residual error is now a function of both and , which has the value minimum of at the optimal point , .

###### 4.2.4. Optimal , and in Case of

Let us consider the optimal approach with the three unknown convergence-control parameters. The corresponding is now a function of , and , which has the minimum at the optimal values , , and .

Based on the above calculations for this equation, we found that all optimal homotopy-analysis approaches can give much better approximations, which converge much faster than those without optimal convergence-control parameters in case of and . But in this example, the first one-parameter optimal homotopy-analysis approach is better than other cases.

The absolute error and comparison of the 15th-order approximate solution and analytic solution (49) with =, = = are shown in Figures 8 and 9, respectively. The absolute errors have been calculated for different cases of , and in Table 7.

#### 5. Conclusion

In this paper, we have successfully developed an optimal homotopy-analysis approach for solving HJB equation arising in nonlinear optimal control problems. With the deformation functions defined, the method used here contains three auxiliary convergence-control parameters, which provide us with a simple way to adjust and control the convergence region of the solution. By minimizing the averaged square residual error, the optimal convergence-control parameters , and can be obtained.

#### References

- B. D. Craven,
*Control and Optimization*, Chapman & Hall, London, UK, 1995. View at MathSciNet - S. Effati and H. Saberi Nik, “Solving a class of linear and non-linear optimal control problems by homotopy perturbation method,”
*IMA Journal of Mathematical Control and Information*, vol. 28, no. 4, pp. 539–553, 2011. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - K. L. Teo, C. J. Goh, and K. H. Wong,
*A Unified Computational Approach to Optimal Control Problems*, Longman Scientific & Technical, Essex, UK, 1991. View at MathSciNet - R. Bellman,
*Dynamic Programming*, Princeton University Press, Princeton, NJ, USA, 1957. View at MathSciNet - R. W. Beard, G. N. Saridis, and J. T. Wen, “Approximate solutions to the time-invariant Hamilton-Jacobi-Bellman equation,”
*Journal of Optimization Theory and Applications*, vol. 96, no. 3, pp. 589–626, 1998. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - J. E. Rubio,
*Control and Optimization, The Linear Treatment of Nonlinear Problems*, Manchester University Press, Manchester, UK, 1986. View at MathSciNet - A. Fakharian, M. T. Hamidi Beheshti, and A. Davari, “Solving the Hamilton-Jacobi-Bellman equation using Adomian decomposition method,”
*International Journal of Computer Mathematics*, vol. 87, no. 12, pp. 2769–2785, 2010. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - H. Saberi Nik, S. Effati, and M. Shirazian, “An approximate-analytical solution for the Hamilton-Jacobi-Bellman equation via homotopy perturbation method,”
*Applied Mathematical Modelling*, vol. 36, no. 11, pp. 5614–5623, 2012. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - S. Effati, H. S. Nik, and M. Shirazian, “An improvement to the homotopy perturbation method for solving the Hamilton Jacobi Bellman equation,”
*IMA Journal of Mathematical Control and Information*, 2013. View at Publisher · View at Google Scholar - H. Saberi Nik, S. Effati, and A. Yildirim, “Solution of linear optimal control systems by differential transform method,”
*Neural Computing and Applications*, 2012. View at Publisher · View at Google Scholar - S. J. Liao, “A kind of approximate solution technique which does not depend upon small parameters. II. An application in fluid mechanics,”
*International Journal of Non-Linear Mechanics*, vol. 32, no. 5, pp. 815–822, 1997. View at Publisher · View at Google Scholar · View at MathSciNet - S. J. Liao, “Comparison between the homotopy analysis method and homotopy perturbation method,”
*Applied Mathematics and Computation*, vol. 169, no. 2, pp. 1186–1194, 2005. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - S. J. Liao, “Homotopy analysis method: a new analytical technique for nonlinear problems,”
*Communications in Nonlinear Science and Numerical Simulation*, vol. 2, no. 2, pp. 95–100, 1997. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at Scopus - S. J. Liao,
*Beyond Perturbation: Introduction to the Homotopy Analysis Method*, CRC Press; Chapman & Hall, Boca Raton, Fla, USA, 2003. View at MathSciNet - T. Hayat, T. Javed, and M. Sajid, “Analytic solution for rotating flow and heat transfer analysis of a third-grade fluid,”
*Acta Mechanica*, vol. 191, no. 3-4, pp. 219–229, 2007. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at Scopus - S. Abbasbandy and F. S. Zakaria, “Soliton solutions for the fifth-order KdV equation with the homotopy analysis method,”
*Nonlinear Dynamics*, vol. 51, no. 1-2, pp. 83–87, 2008. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at Scopus - S.-P. Zhu, “An exact and explicit solution for the valuation of American put options,”
*Quantitative Finance*, vol. 6, no. 3, pp. 229–242, 2006. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - S. J. Liao, “An optimal homotopy-analysis approach for strongly nonlinear differential equations,”
*Communications in Nonlinear Science and Numerical Simulation*, vol. 15, no. 8, pp. 2003–2016, 2010. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - S. J. Liao,
*Homotopy Analysis Method in Nonlinear Differential Equations*, Springer, Berlin, Germany, 2012. - W. H. Fleming and R. W. Rishel,
*Deterministic and Stochastic Optimal Control*, Springer, New York, NY, USA, 1975. View at MathSciNet - M. R. Caputo,
*Foundations of Dynamic Economic Analysis: Optimal Control Theory and Applications*, Cambridge University Press, Cambridge, UK, 2005. View at MathSciNet