Research Article | Open Access

# Optimal Control with Time Delays via the Penalty Method

**Academic Editor:**Ligang Wu

#### Abstract

We prove necessary optimality conditions of Euler-Lagrange type for a problem of the calculus of variations with time delays, where the delay in the unknown function is different from the delay in its derivative. Then, a more general optimal control problem with time delays is considered. Main result gives a convergence theorem, allowing us to obtain a solution to the delayed optimal control problem by considering a sequence of delayed problems of the calculus of variations.

#### 1. Introduction

Over the past years, there has been an increasing interest in time delay problems of the calculus of variations and control [1–4]. Such interest is explained for their importance in control and engineering [5–8]. Indeed, time delays are inherent in various real systems, such as control systems and optimal control problems in engineering [9, 10].

In this paper we improve recent optimality conditions for time delay variational problems. In [11] necessary optimality conditions of Euler-Lagrange, DuBois-Reymond, and Noether type were obtained for problems of the calculus of variations with a time delay. The results of [11] were then extended to delayed variational problems with higher order derivatives in [2]. Here we model time delay variational problems in a more realistic way: while in [2, 11] the delay on functions and their derivatives (and control variables) is always the same, here we consider different delays for the functions and derivatives/controls.

The text is organized as follows. In Section 2 we formulate the delayed problem of the calculus of variations, where the delay in the unknown functions is different from the delay in their derivatives. The main result in this section is Theorem 4, which provides necessary optimality conditions of Euler-Lagrange type. Control strategies via an exterior penalty method are then investigated in Section 3. The idea is to replace the optimal control problem with time delays by a series of delayed problems of the calculus of variations. The main result gives a convergence theorem that allows us to obtain a solution to delayed optimal control problems with linear delayed control systems, by considering a sequence of variational problems with time delays of the type considered before in Section 2 (see Theorem 7). We end with Section 4 of conclusions.

#### 2. Calculus of Variations with Time Delays

We consider the following fundamental problem of the calculus of variations with time delays, where the delay in the function we are looking for is different from the delay in its derivative: subject to where , , is the Lagrangian, is fixed in , and are two given positive real numbers such that , and and are given piecewise smooth functions. Let , be the Lebesgue space of measurable functions such that and the Sobolev space of functions having their weak first derivative lying in and represented by for all and in . We denote that(i) is the space of all functions such that , , and , which is a Hilbert space with the norm (ii);(iii) is the functional

Our problem (1) and (2) takes then the following form: We make the following assumptions on the data of problem (7): Lagrangian is a Carathéodory mapping; that is, it is of class in for almost all and is measurable in for every ;there exist , , such that a.e. in where is the partial derivative of with respect to its th argument.

*Definition 1 (cone of tangents). *Let be a normed space, , and . The cone of tangents is the set of all with the property that there is a sequence in converging strongly to and a sequence of nonnegative numbers such that .

Lemma 2. *The set is an affine linear subspace of and the cone of tangents is given by
*

*Proof. *Let . Then there exist and such that in implies that in . Since for all , we have
Hence,
Therefore, with for all , for almost all , and . Thus,
Conversely, let for . Define . Then with . Hence, .

For convenience, we introduce the operator defined by

Proposition 3. *Under conditions and , the mapping is Fréchet differentiable and
*

*Proof. *Let . We have
Define
and
Then, as for almost all . On the other hand,
a.e. in with
a function not depending on , and
for almost all and sufficiently small. Since has finite measure, Lebesgue’s theorem yields that
as . Hence,
This is the directional derivative of in the direction . To finish the proof, we need to show that is linear and bounded in and continuous in . The linearity is obvious. We begin by proving that is bounded from to :
We still need to prove the continuity of . Let in . Then,
where
On the other hand, in . From Lebesgue’s theorem, there exists such that and
Hence,
Since is -Carathéodory, assumption assures from Lebesgue’s theorem that
This implies that . Then, . The proof is complete.

Theorem 4 (necessary optimality conditions of Euler-Lagrange type for problem (1) and (2)). *Under conditions and , if is a minimizer to problem (1) and (2), then satisfies the following Euler-Lagrange equations with time delay:
*

*Proof. *If is a minimizer to problem (1) and (2), then
for all ; that is,
for all with , . Integration by parts yields
By (31) and (32), we obtain that
for all . On the other hand,
Hence,
for all . Put
Then,
for all . In particular, for such that for almost all and for almost all , we have
or
The proof is complete.

#### 3. Optimal Control with Time Delays

Now we prove existence of an optimal solution to more general problems of optimal control with time delays. The result is obtained via the exterior penalty method [12, 13] and Theorem 4. The optimal control problem with time delays is defined as follows: subject to where , , is an matrix, is an matrix, and , . The final time is fixed in , and are two given positive real numbers such that , and, as before, and are given piecewise smooth functions. In the sequel, we denote by the function defined by , , and , . We make the following assumptions on the data of the problem. The mapping is a -Carathéodory mapping; that is, is in for almost all and is measurable in for every .There exist , such that where is the partial derivative of with respect to its th argument, .There exists such that for almost all and for all is convex in .

Using the exterior penalty function method, we consider the following sequence of unconstrained optimal control problems corresponding to (40) and (41): where , . Denote The sequence of unconstrained optimal control problems takes then the following form: .

Lemma 5. *The cone of tangents is given by
*

*Proof. *It is similar to the proof of Lemma 2.

It is well known that the penalty function method is a very effective technique for solving constrained optimization problems via unconstrained ones. The main question is the convergence of the sequence of solutions of the unconstrained optimal control problems to the original/constrained problem. Before giving the convergence theorem, we begin with some preparatory results, which are a direct consequence of the necessary optimality conditions given by Theorem 4.

Proposition 6. *For every , if is an optimal solution to , then *(i)* **where
*(ii)*there exists such that for all and all sufficiently large.*

*Proof. *(i) Let be an optimal solution to . Then, by Lemmas 2 and 5 and Theorem 4, we obtain the necessary conditions of item (i) for problem .

(ii) Since and exists, is defined and there exists such that . By the first equation of item (i), we have
Consequently,
for all with , , and . By Gronwall’s lemma, we obtain that
The second and third equalities of item (i) give
Now, inequalities (50) and (51) imply that
with
Since , there exists such that
for all and for all large.

We are now ready to prove the convergence theorem, which reads as follows.

Theorem 7 (penalty convergence theorem). *If hypotheses hold and problem (40) and (41) has a finite value, then the sequence of solutions to contains a subsequence such that *(i)* strongly in ;*(ii)* weakly in ;*(iii)* weakly in ;**with a solution to problem (40) and (41).*

*Proof. *Let be an optimal solution to for every . By Proposition 6,
Because
it follows that
On the other hand, if denotes the finite value of (40) and (41), then
By assumption , there exists such that
Thus,
By Gronwall’s lemma, we obtain that
where
Similarly, for sufficiently large,
For all , we have
Since and is bounded in , with of finite measure, there exists such that
For all we have
As before, we can assert that
By (65) and (67), there exists such that
in for sufficiently large. Therefore, there exists a subsequence of converging to . Since for all , by the use of (61), the sequence is equi-bounded and equi-continuous (because is bounded in ). Ascoli’s theorem implies that
Since
we obtain that and a.e. . The sequence is bounded in . Thus, there exists a subsequence such that weakly in . To complete the proof, we show that is an optimal solution to (40)-(41). By Proposition 6, we have
Hence,
with . We conclude that
On the other hand,
Consequently,
This implies that
Thus,
and a.e. , . Then, is an admissible pair and
On the other hand,
Now the hypotheses , , and , together with Lebesgue’s theorem, assert that
that is,
This implies that the pair is a solution to problem (40) and (41).

#### 4. Conclusion

New optimality conditions for problems of the calculus of variations and optimal control with time delays, where the delay in the unknown function differs from the delay in its derivative/control, were obtained. The proofs are first given in the simpler context of the delayed calculus of variations and then extended to delayed optimal control problems by using a penalty method. New results include a convergence theorem (see Theorem 7), which is of great practical interest because it allows us to obtain a solution to a delayed optimal control problem by considering a sequence of simpler problems of the calculus of variations. Previous results in the literature [2, 3, 11] consider the delay in the unknown function to be the same as the delay in its derivative. There is, however, no justification for the delays to be the same. In contrast with those results, here we consider the case of multiple time delays. Moreover, the procedure of our proofs is completely different from the case of one time delay only, which relies on the Lagrange multiplier method. Such approach introduces a new unknown function, the Lagrange multiplier, for which it is hard to set the interpolation space. Indeed, the Lagrange multiplier must be carefully selected in order to be possible to obtain an accurate solution. Otherwise, the resulting system of equations may become singular, in particular if the number of degrees of freedom is too large. Here we use a penalty method, which requires only the choice of one scalar parameter. Big values of this parameter are used in order to impose the boundary conditions in a proper manner. Furthermore, in our case the use of the penalty method replaces a constrained optimization problem (the delayed optimal control problem) by a sequence of unconstrained problems of the calculus of variations with time delay whose solutions converge to the solution of the original constrained problem. Similarly to [11], our results can be easily extended for controls with time delay.

#### Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

#### Acknowledgments

This work was partially supported by Portuguese funds through the* Center for Research and Development in Mathematics and Applications* (CIDMA) and* The Portuguese Foundation for Science and Technology* (FCT), within Project PEst-OE/MAT/UI4106/2014. Torres was also supported by the FCT Project PTDC/EEI-AUT/1450/2012, cofinanced by FEDER under POFC-QREN with COMPETE reference FCOMP-01-0124-FEDER-028894. The authors are grateful to two anonymous referees for valuable remarks and comments, which significantly contributed to the quality of the paper.

#### References

- C. Dacka, “On the controllability of nonlinear systems with time-variable delays,”
*IEEE Transactions on Automatic Control*, vol. 26, no. 4, part 1, pp. 956–959, 1981. View at: Publisher Site | Google Scholar | MathSciNet - G. S. F. Frederico, T. Odzijewicz, and D. F. M. Torres, “Noether's theorem for non-smooth extremals of variational problems with time delay,”
*Applicable Analysis.*, vol. 93, no. 1, pp. 153–170, 2014. View at: Publisher Site | Google Scholar | MathSciNet - G. S. F. Frederico and D. F. M. Torres, “A nondifferentiable quantum variational embedding in presence of time delays,”
*International Journal of Difference Equations*, vol. 8, no. 1, pp. 49–62, 2013. View at: Google Scholar | MathSciNet - J.-Q. Sun and Q. Ding,
*Advances in Analysis and Control of Time-Delayed Dynamical Systems*, World Scientific Publishing, Singapore, 2013. - A. Debbouche and D. F. M. Torres, “Approximate controllability of fractional nonlocal delay semilinear systems in Hilbert spaces,”
*International Journal of Control*, vol. 86, no. 9, pp. 1577–1585, 2013. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - A. Debbouche and D. F. M. Torres, “Approximate controllability of fractional delay dynamic inclusions with nonlocal control conditions,”
*Applied Mathematics and Computation*, vol. 243, pp. 161–175, 2014. View at: Publisher Site | Google Scholar - J.-P. Richard, “Time-delay systems: an overview of some recent advances and open problems,”
*Automatica*, vol. 39, no. 10, pp. 1667–1694, 2003. View at: Publisher Site | Google Scholar | MathSciNet - D. Salamon, “On controllability and observability of time delay systems,”
*IEEE Transactions on Automatic Control*, vol. 29, no. 5, pp. 432–439, 1984. View at: Publisher Site | Google Scholar | MathSciNet - L. Göllmann, D. Kern, and H. Maurer, “Optimal control problems with delays in state and control variables subject to mixed control-state constraints,”
*Optimal Control Applications & Methods*, vol. 30, no. 4, pp. 341–365, 2009. View at: Publisher Site | Google Scholar | MathSciNet - L. Göllmann and H. Maurer, “Theory and applications of optimal control problems with multiple time-delays,”
*Journal of Industrial and Management Optimization*, vol. 10, no. 2, pp. 413–441, 2014. View at: Google Scholar | MathSciNet - G. S. F. Frederico and D. F. M. Torres, “Noether's symmetry theorem for variational and optimal control problems with time delay,”
*Numerical Algebra, Control and Optimization*, vol. 2, no. 3, pp. 619–630, 2012. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - M. Benharrat and H. Mokhtar-Kharroubi, “Exterior penalty in optimal control problem with state-control constraints,”
*Rendiconti del Circolo Matematico di Palermo*, vol. 59, no. 3, pp. 389–403, 2010. View at: Publisher Site | Google Scholar | MathSciNet - S. Y. Serovaiskii, “An approximate penalty method in the problem of the optimal control of nonsmooth singular systems,”
*Mathematical Notes*, vol. 76, no. 5-6, pp. 834–843, 2004. View at: Publisher Site | Google Scholar

#### Copyright

Copyright © 2014 Mohammed Benharrat and Delfim F. M. Torres. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.