Abstract and Applied Analysis

Volume 2011 (2011), Article ID 486714, 18 pages

http://dx.doi.org/10.1155/2011/486714

## The Optimization of Solutions of the Dynamic Systems with Random Structure

^{1}University of Žilina, Žilina, Slovakia^{2}Vadim Getman Kyjew National Economic University, Kyjew, Ukraine

Received 31 January 2011; Accepted 31 March 2011

Academic Editor: Josef Diblík

Copyright © 2011 Miroslava Růžičková and Irada Dzhalladova. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

The paper deals with the class of jump control systems with semi-Markov coefficients. The control system is described as the system of linear differential equations. Every jump of the random process implies the random transformation of solutions of the considered system. Relations determining the optimal control to minimize the functional are derived using Lyapunov functions. Necessary conditions of optimization which enables the synthesis of the optimal control are established as well.

#### 1. The Statement of the Problem

The optimal control theory as mathematical optimization method for deriving control policies plays an important role in the development of the modern mathematical control theory. The optimal control deals with the problem of finding such a control law for a given system that a certain optimality criterion is achieved. The background for the optimization method can be found in the work of Lev Pontryagin with his well-known Pontryagin's maximum principle. The optimal control has been applied in diverse fields, such as economics, bioengineering, process control, and many others. Some real-life problems are described by a continuous-time or discrete-time linear system of differential equations, but a lot of them are described by dynamic systems with random jumping changes, for example economics systems. The general theory of random structure systems can be found in the work of Artemiev and Kazakov [1]. The optimization of linear systems with random parameters are considered in many works, for example in [2–12]. Particularly, the original results concerning the stabilization of the systems with random coefficients and a random process are derived using moment equations and Lyapunov functions in [4]. These results create a more convenient technique for applying the method in practice using suitable software for engineering or economics investigation. Our aim is the expansion of the achieved results to a new class of systems of linear differential equations with semi-Markov coefficients and random transformation of solutions performed simultaneously with jumps of semi-Markov process. We will focus on using the particular values of Lyapunov functions for the calculation of coefficients of the control vector which minimize the quality criterion. We will also establish the necessary conditions of the optimal solution which enables the synthesis of the optimal control for the considered class of systems.

Let us consider the linear control system on the probability basis and together with (1.1) we consider the initial conditions The coefficients of the system are semi-Markov coefficients defined by the transition intensities , from state to state . We suppose that the vectors belong to the set of control and the functions , satisfy the conditions [13]:

*Definition 1.1. *Let the matrices , with semi-Markov elements be symmetric and positive definite. The cost functional
defined on the space , where denotes mathematical expectation, is called *the quality criterion*.

*Definition 1.2. *Let be a matrix with semi-Markov elements. The control vector
which minimizes the quality criterion with respect to the system (1.1) is called *the optimal control*.

If we denote then the system (1.1) can be rewritten to the form and the functional (1.4) to the form

We suppose also that, together with every jump of random process in time , the solutions of the system (1.7) submit to the random transformation if the conditions , hold.

*Definition 1.3. *Let , , be a selection of different positive functions. If , , , and for the equality holds, then the function is called *semi-Markov function*.

The application of semi-Markov functions makes it possible to use the concept of stochastic operator. In fact, the semi-Markov function is an operator of the semi-Markov process , because the value of the semi-Markov function is defined not only by the values and , but it is also necessary to specify the function , and the value of the jump of the process in time which precedes the moment of time .

Our task is the construction of Lyapunov function for the new class of systems of linear differential equations with semi-Markov coefficients and then applying the function to solve the optimization problem which minimizes the quality criterion.

#### 2. Auxiliary Results

In the proof of Theorem 3.1 in Section 3, we will employ two results concerning the construction of the Lyapunov function and the construction of the optimal control for the system of linear differential equations in a deterministic case. We will derive these auxiliary results in this part.

##### 2.1. The Construction of the Lyapunov Function

Let us consider the system of linear differential equations associated to the system (1.1).

Let us define a quadratic form where elements of the matrix are the semi-Markov processes. The matrix is defined by such a set of different symmetric and positive definite matrices , , , that the equality for implies

Our purpose in this section is to express the value of the functional in a convenient form, which can help us to prove the -stability of the trivial solution of the system (2.1).

At first, we introduce the particular Lyapunov functions If we can find the values of the particular Lyapunov functions in the form , , then value of the functional can be expressed by the formula where the scalar value is called the scalar product of the two matrices , and has the property [14]

The first auxiliary result contains two equivalent, necessary, and sufficient conditions for the -stability (see in [4]) of the trivial solution of the system (2.1) and one sufficient condition for the stability of the solutions.

Theorem 2.1. *The trivial solution of the system (2.1) is -stable if and only if any of the next two equivalent conditions hold: *(1)*the system of equations
has a solution for , , *(2)*the sequence of the approximations
converges.** **Moreover, the solutions of the system (2.1) are -stabile, if there exist symmetric and positive definite matrices , such that the property
**
holds.*

*Proof. *We will construct a system of equations, which will define the particular Lyapunov functions , . Let us introduce the auxiliary semi-Markov functions
For the state of the random process , the equalities
are true. Simultaneously, with the jumps of the random process , the jumps of solutions of (2.1) occurred, so in view of (2.12), we derive the equations
Further, if we introduce denoting
then (2.14) can be rewritten as the system of integral equations for the matrix in the form
We define matrices and functions , with regard to (2.5) and (2.12), by formulas
Integrating the system (2.16) from 0 to , we get the system
Similarly, integrating the system of (2.14), we get the system of equations determining the particular Lyapunov functions
Let us denote
If there exist such positive constants that
or equivalent conditions
hold, then the matrices , are symmetric and positive definite. Using (2.17), the system (2.18) can be rewritten to the form
It is easy to see that the system (2.23) is conjugated to the system (2.9). Therefore, the existence of a positive definite solution of the system (2.23) is equivalent to the existence of a positive definite solution and it is equivalent to -stability of the solution of the system (2.1). On the other hand, if the existence of the particular Lyapunov functions in (2.5) implies -stability of the solutions of the system (2.1), then, in view of conditions (2.22) and the convergence of the integral (2.17), we get the inequality
The theorem is proved.

*Remark 2.2. *If the system of linear differential equations (2.1) is a system with piecewise constant coefficients and the function has the form
then the system (2.18) can be written in the form

Particularly, if the semi-Markov process is identical with a Markov process, then the system (2.26) has the form or, more simply Moreover, under the assumption that the integral in (2.28) converges, the system (2.28) is equivalent to the system of matrices equations which can be written as the system if .

*Example 2.3. *Let the semi-Markov process take two states , and let it be identical with the Markov process described by the system of differential equations
We will consider the -stability of the solutions of the differential equation
constructing a system of the type (2.26) related to (2.32). The system is
and its solution is
The trivial solution of (2.32) is -stable, if and . Let the intensities of semi-Markov process satisfy the conditions
Then, using the Theorem 2.1, the conditions
are sufficient conditions for the -stability of solutions of (2.32).

##### 2.2. The Construction of an Optimal Control for the System of Linear Differential Equations in the Deterministic Case

Let us consider the deterministic system of the linear equations in the boundary field , where , and together with (2.37) we consider the initial conditions We assume that the vector belongs to the control set . The quality criterion has the form of the quadratic functional in the space . The control vector which minimizes the quality criterion (2.39) is called the optimal control.

The optimization problem is the problem of finding the optimal control (2.40) from all feasible control , or, in fact, it is the problem of finding the equation to determine , .

Theorem 2.4. *Let there exist the optimal control (2.40) for the system of (2.37). Then the control equations
**
where the matrix satisfies the Riccati equation
**
determines the synthesis of the optimal control.*

*Proof. *Let the control for the system (2.37) have the form (2.40), where the matrix is unknown. Then, the minimum value of the quality criterion (2.39) is
Under assumption that the vector is known and using Pontryagin's maximum principle [1, 15], the minimum of the quality criterion (2.39) is written as
where
is the row-vector. If we take Hamiltonian function [15] of the form
the necessary condition for optimality is
where are elements of the matrix . The scalar value
is called derivative of the matrix with respect to the matrix .

Employing the scalar product of the two matrices in our calculation, the Hamiltonian function (2.46) can be rewritten into the form
and its derivative with respect to the matrix is
Because the equality (2.50) holds for any value of , the expression of the vector control has the form
which implies
If we put the expression of matrix to (2.49), we obtain a new expression for the Hamiltonian function
for which the canonical system of linear differential equations
has the form

In the end, we define the matrix as the integral manifolds of solutions of the system equations
If we derive the system (2.56) with respect to regarding the system (2.55) and extract the vector , then we obtain the matrix differential equation (2.40). This equation is known as Riccati equation in literature, see for example in [16, 17]. The solution of (2.42) satisfying the initial condition
determines the minimum of the functional
and can be obtained as the limit of the sequence of the successive approximations :

*Remark 2.5. *Similar results can be obtained from the Bellman equation [18], where the function satisfie

#### 3. The Main Result

Theorem 3.1. *Let the coefficients of the control system (1.1) be the semi-Markov functions and let them be defined by the equations
**
Then the set of the optimal control is a nonempty subset of the control , which is identical with the family of the solutions of the system
**
where the matrix is defined by the system of Riccati type of differential equations
*

##### 3.1. The Proof of Main Result Using Lyapunov Functions

It should be recalled that the coefficients of the systems (1.1), (1.7) and of the functionals (1.4), (1.8) have the form if , . In addition to this, we have The formula is useful for the calculation of the particular Lyapunov functions , of the functional (1.8). We get or, the more convenient form Then the system (3.1) has the form Let us assume that for the control system (1.1) the optimal control exists in the form (1.5) independent of the initial value . Regarding the formula (3.6), there exist minimal values of the particular Lyapunov functions , , which are associated with the optimal control. It also follows from the fact that the functions , are particular values of the functional (3.6). Finding the minimal values , by choosing the optimal control is a well-studied problem, for the main results see [16]. It is significant that all matrices , of the integrand in the formula (3.8) are constant matrices, hence, solving the optimization problem they can be considered as matrices of parameters.

Therefore, the problem to find the optimal control (1.5) for the system (1.1) can be transformed to problems to find the optimal control for the deterministic system (3.9), which is equivalent to the system of linear differential equations of type (2.37).

##### 3.2. The Proof of the Main Result Using Lagrange Functions

In this part, we get one more proof of the Theorem 3.1 using the Lagrange function.

We are looking for the optimal control which reaches the minimum of quality criterion Let us introduce the Lagrange function where is the column-vector of Lagrange multipliers. In accordance with Pontryagin's maximum principle, we put the first variations of the functionals , equal to zero and we obtain the system of linear differential equations Then the optimal control can be expressed by The synthesis of the optimal control needs to find the integral manifolds of the solutions of the system (3.12) in the form According to the theory of integral manifolds [19] we construct the differential matrix equations of the Riccati type for the matrix . Integrating them from time to time and using the initial condition we obtain Lagrange functions for the optimal control We will prove that Differentiating the equality (3.17) with respect to we obtain the matrix equation and extracting the optimal control we obtain differential equation for identical with (3.15). The equality follows from the positive definite matrices , for . Therefore, from (3.17) we get ; moreover, from (3.10) it follows that . Applying the formulas (3.15), (3.16) to the system (3.8) with minimal functionals (3.9), the expression for the optimal control can be found in the form where symmetric matrices satisfy the matrix system of differential equations The systems (3.9), (3.20) define the necessary condition such that the solutions of the systems (1.4) will be optimal. In addition to this, the system (3.8) defines the matrices , of the optimal control in the form We define matrices from the system equations (3.20) in the view of In regards to it can makes the system (3.20) simpler. Then the system (3.20) takes the form (3.3), and formula (3.2) defines the optimal control.

*Remark 3.2. *If the control system (1.1) is deterministic, then , and the system (3.3) is identical to the system of the Riccati type equations (3.15).

#### 4. Particular Cases

The optimal control for the system (1.1) has some special properties, and the equations determining it are different from those given in the previous section in case the coefficients of the control system (1.1) have special properties or intensities satisfy some relations or some other special conditions are satisfied. Some of these cases will be formulated as corollaries.

Corollary 4.1. *Let the control system (1.1) with piecewise constant coefficients have the form
**
Then the quadratics functional
**
determines the optimal control in the form
**
where
**
and the matrices satisfy the equations
**
if , . **The matrices , are the solutions of the systems of the Riccati-type equations:
*

*Remark 4.2. *In the corollary we mention piecewise constant coefficients of the control system (4.1). The coefficients of the functional (4.2) will be piecewise as well, but the optimal control is nonstationary.

Corollary 4.3. *Assume that
**
Then the optimal control will be piecewise constant.*

Taking into consideration that the optimal control is piecewise constant, we find out that the matrices in (4.5) are constant, which implies the form of the system (4.6) is changed to the form The system (4.8) has constant solutions , if conditions (4.7) hold. Moreover, if the random process is a Markov process then the conditions (4.7) have the form and the system (4.8) transforms to the form for which the optimal control is

Corollary 4.4. *Let the state of the semi-Markov process be no longer than . Then the system (3.8) has the form
**
Because
**
then
*

In this case, the search for the matrix in concrete tasks is reduced to integration of the matrix system of differential equations (3.15) on the interval with initial conditions (4.14). In view of , we can expect that every equation (3.15) has a singular point . If has simple zero at the point , then the system (4.6) meets the necessary condition for boundary of matrix in the singular points.

#### Acknowledgments

The authors would like to thank the following for their support: the Slovak Research and Development Agency (Project APPV-0700-07), the Grant Agency of the Slovak republic (VEGA1/0090/09), and the National Scholarship Program of the Slovak Republic (SAIA).

#### References

- V. M. Artemiev and I. E. Kazakov,
*Handbook on the Theory of Automatic Control*, Nauka, Moscow, Russia, 1987. - K.G. Valeev and I. A. Dzhalladova,
*Optimization of Random Process*, KNEU, Kyjew, Russia, 2006. - I. I. Gihman and A. V. Skorohod,
*Controlable of Random Process*, Izdat. “Naukova Dumka”, Kyjew, Russia, 1977. - I. A. Dzhalladova,
*Optimization of Stochastic System*, KNEU, Kyjew, Russia, 2005. - V. K. Jasinskiy and E. V. Jasinskiy,
*Problem of Stability and Stabilization of Dynamic Systems with Finite after Effect*, TVIMS, Kyjew, Russia, 2005. - K. J. Astrom,
*Introduction to Stochastic Control Theory*, vol. 70 of*Mathematics in Science and Engineering*, Academic Press, New York, NY, USA, 1970. - R. Balaji,
*Introduction to Stochastic Finance*, University to Conecticut, Academic Press, New York, NY, USA, 1997. - J. K. Hale and S. M. Verduyn Lunel,
*Introduction to Functional-Differential Equations*, vol. 99 of*Applied Mathematical Sciences*, Springer, New York, NY, USA, 1993. - O. Hájek,
*Control Theory in the Plane*, vol. 153 of*Lecture Notes in Control and Information Sciences*, Springer, Berlin, Germany, 1991. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - X. Liao and P. Yu,
*Absolute Stability of Nonlinear Control Systems*, vol. 25 of*Mathematical Modelling: Theory and Applications*, Springer, New York, NY, USA, Second edition, 2008. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - G. D. Qushner,
*Stochastic Stability and Control*, Mir, Moscow, Russia, 1969. - L. Glass and M. C. Mackey,
*From Clocks to Chaos. The Rythms of Life*, Princeton University Press, Princeton, NJ, USA, 1988. View at Zentralblatt MATH - K.G. Valeev and O. L. Strijak,
*Methods of Moment Equations*, AN USSR, Kyjew, Russia, 1985. - K. G. Valeev, O. L. Karelova, and V. I. Gorelov,
*Optimization of a System of Linear Differential Equations with Random Coefficients*, RUDN, Moscow, Russia, 1996. - H. Kwakernaak and R. Sivan,
*Linear Optimal Control Systems*, John Wiley & Sons, New York, NY, USA, 1972. View at Zentralblatt MATH - K.G. Valeev and G. S. Finin,
*Constructing of Lyapunov Function*, Naukova dumka, Kyjew, Russia, 1981. - E. A. Barbashyn,
*Lyapunov Functions*, Nauka, Moscow, Russia, 1970. - R. Bellman,
*Introduction to Matrix Analysis*, McGraw-Hill, New York, NY, USA, 1960. View at Zentralblatt MATH - K. G. Valeev and O. A. Jautykov,
*Infinite Systems of Differential Equations*, Nauka, Alma-Ata, Russia, 1974.