- About this Journal
- Abstracting and Indexing
- Aims and Scope
- Annual Issues
- Article Processing Charges
- Articles in Press
- Author Guidelines
- Bibliographic Information
- Citations to this Journal
- Contact Information
- Editorial Board
- Editorial Workflow
- Free eTOC Alerts
- Publication Ethics
- Reviewers Acknowledgment
- Submit a Manuscript
- Subscription Information
- Table of Contents
Abstract and Applied Analysis
Volume 2011 (2011), Article ID 486714, 18 pages
The Optimization of Solutions of the Dynamic Systems with Random Structure
1University of Žilina, Žilina, Slovakia
2Vadim Getman Kyjew National Economic University, Kyjew, Ukraine
Received 31 January 2011; Accepted 31 March 2011
Academic Editor: Josef Diblík
Copyright © 2011 Miroslava Růžičková and Irada Dzhalladova. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
The paper deals with the class of jump control systems with semi-Markov coefficients. The control system is described as the system of linear differential equations. Every jump of the random process implies the random transformation of solutions of the considered system. Relations determining the optimal control to minimize the functional are derived using Lyapunov functions. Necessary conditions of optimization which enables the synthesis of the optimal control are established as well.
1. The Statement of the Problem
The optimal control theory as mathematical optimization method for deriving control policies plays an important role in the development of the modern mathematical control theory. The optimal control deals with the problem of finding such a control law for a given system that a certain optimality criterion is achieved. The background for the optimization method can be found in the work of Lev Pontryagin with his well-known Pontryagin's maximum principle. The optimal control has been applied in diverse fields, such as economics, bioengineering, process control, and many others. Some real-life problems are described by a continuous-time or discrete-time linear system of differential equations, but a lot of them are described by dynamic systems with random jumping changes, for example economics systems. The general theory of random structure systems can be found in the work of Artemiev and Kazakov . The optimization of linear systems with random parameters are considered in many works, for example in [2–12]. Particularly, the original results concerning the stabilization of the systems with random coefficients and a random process are derived using moment equations and Lyapunov functions in . These results create a more convenient technique for applying the method in practice using suitable software for engineering or economics investigation. Our aim is the expansion of the achieved results to a new class of systems of linear differential equations with semi-Markov coefficients and random transformation of solutions performed simultaneously with jumps of semi-Markov process. We will focus on using the particular values of Lyapunov functions for the calculation of coefficients of the control vector which minimize the quality criterion. We will also establish the necessary conditions of the optimal solution which enables the synthesis of the optimal control for the considered class of systems.
Let us consider the linear control system on the probability basis and together with (1.1) we consider the initial conditions The coefficients of the system are semi-Markov coefficients defined by the transition intensities , from state to state . We suppose that the vectors belong to the set of control and the functions , satisfy the conditions :
Definition 1.1. Let the matrices , with semi-Markov elements be symmetric and positive definite. The cost functional defined on the space , where denotes mathematical expectation, is called the quality criterion.
Definition 1.2. Let be a matrix with semi-Markov elements. The control vector which minimizes the quality criterion with respect to the system (1.1) is called the optimal control.
We suppose also that, together with every jump of random process in time , the solutions of the system (1.7) submit to the random transformation if the conditions , hold.
Definition 1.3. Let , , be a selection of different positive functions. If , , , and for the equality holds, then the function is called semi-Markov function.
The application of semi-Markov functions makes it possible to use the concept of stochastic operator. In fact, the semi-Markov function is an operator of the semi-Markov process , because the value of the semi-Markov function is defined not only by the values and , but it is also necessary to specify the function , and the value of the jump of the process in time which precedes the moment of time .
Our task is the construction of Lyapunov function for the new class of systems of linear differential equations with semi-Markov coefficients and then applying the function to solve the optimization problem which minimizes the quality criterion.
2. Auxiliary Results
In the proof of Theorem 3.1 in Section 3, we will employ two results concerning the construction of the Lyapunov function and the construction of the optimal control for the system of linear differential equations in a deterministic case. We will derive these auxiliary results in this part.
2.1. The Construction of the Lyapunov Function
Let us consider the system of linear differential equations associated to the system (1.1).
Let us define a quadratic form where elements of the matrix are the semi-Markov processes. The matrix is defined by such a set of different symmetric and positive definite matrices , , , that the equality for implies
Our purpose in this section is to express the value of the functional in a convenient form, which can help us to prove the -stability of the trivial solution of the system (2.1).
At first, we introduce the particular Lyapunov functions If we can find the values of the particular Lyapunov functions in the form , , then value of the functional can be expressed by the formula where the scalar value is called the scalar product of the two matrices , and has the property 
The first auxiliary result contains two equivalent, necessary, and sufficient conditions for the -stability (see in ) of the trivial solution of the system (2.1) and one sufficient condition for the stability of the solutions.
Theorem 2.1. The trivial solution of the system (2.1) is -stable if and only if any of the next two equivalent conditions hold: (1)the system of equations
has a solution for , , (2)the sequence of the approximations
Moreover, the solutions of the system (2.1) are -stabile, if there exist symmetric and positive definite matrices , such that the property holds.
Proof. We will construct a system of equations, which will define the particular Lyapunov functions , . Let us introduce the auxiliary semi-Markov functions For the state of the random process , the equalities are true. Simultaneously, with the jumps of the random process , the jumps of solutions of (2.1) occurred, so in view of (2.12), we derive the equations Further, if we introduce denoting then (2.14) can be rewritten as the system of integral equations for the matrix in the form We define matrices and functions , with regard to (2.5) and (2.12), by formulas Integrating the system (2.16) from 0 to , we get the system Similarly, integrating the system of (2.14), we get the system of equations determining the particular Lyapunov functions Let us denote If there exist such positive constants that or equivalent conditions hold, then the matrices , are symmetric and positive definite. Using (2.17), the system (2.18) can be rewritten to the form It is easy to see that the system (2.23) is conjugated to the system (2.9). Therefore, the existence of a positive definite solution of the system (2.23) is equivalent to the existence of a positive definite solution and it is equivalent to -stability of the solution of the system (2.1). On the other hand, if the existence of the particular Lyapunov functions in (2.5) implies -stability of the solutions of the system (2.1), then, in view of conditions (2.22) and the convergence of the integral (2.17), we get the inequality The theorem is proved.
Particularly, if the semi-Markov process is identical with a Markov process, then the system (2.26) has the form or, more simply Moreover, under the assumption that the integral in (2.28) converges, the system (2.28) is equivalent to the system of matrices equations which can be written as the system if .
Example 2.3. Let the semi-Markov process take two states , and let it be identical with the Markov process described by the system of differential equations We will consider the -stability of the solutions of the differential equation constructing a system of the type (2.26) related to (2.32). The system is and its solution is The trivial solution of (2.32) is -stable, if and . Let the intensities of semi-Markov process satisfy the conditions Then, using the Theorem 2.1, the conditions are sufficient conditions for the -stability of solutions of (2.32).
2.2. The Construction of an Optimal Control for the System of Linear Differential Equations in the Deterministic Case
Let us consider the deterministic system of the linear equations in the boundary field , where , and together with (2.37) we consider the initial conditions We assume that the vector belongs to the control set . The quality criterion has the form of the quadratic functional in the space . The control vector which minimizes the quality criterion (2.39) is called the optimal control.
The optimization problem is the problem of finding the optimal control (2.40) from all feasible control , or, in fact, it is the problem of finding the equation to determine , .
Theorem 2.4. Let there exist the optimal control (2.40) for the system of (2.37). Then the control equations where the matrix satisfies the Riccati equation determines the synthesis of the optimal control.
Proof. Let the control for the system (2.37) have the form (2.40), where the matrix is unknown. Then, the minimum value of the quality criterion (2.39) is
Under assumption that the vector is known and using Pontryagin's maximum principle [1, 15], the minimum of the quality criterion (2.39) is written as
is the row-vector. If we take Hamiltonian function  of the form
the necessary condition for optimality is
where are elements of the matrix . The scalar value
is called derivative of the matrix with respect to the matrix .
Employing the scalar product of the two matrices in our calculation, the Hamiltonian function (2.46) can be rewritten into the form and its derivative with respect to the matrix is Because the equality (2.50) holds for any value of , the expression of the vector control has the form which implies If we put the expression of matrix to (2.49), we obtain a new expression for the Hamiltonian function for which the canonical system of linear differential equations has the form
In the end, we define the matrix as the integral manifolds of solutions of the system equations If we derive the system (2.56) with respect to regarding the system (2.55) and extract the vector , then we obtain the matrix differential equation (2.40). This equation is known as Riccati equation in literature, see for example in [16, 17]. The solution of (2.42) satisfying the initial condition determines the minimum of the functional and can be obtained as the limit of the sequence of the successive approximations :
Remark 2.5. Similar results can be obtained from the Bellman equation , where the function satisfie
3. The Main Result
Theorem 3.1. Let the coefficients of the control system (1.1) be the semi-Markov functions and let them be defined by the equations Then the set of the optimal control is a nonempty subset of the control , which is identical with the family of the solutions of the system where the matrix is defined by the system of Riccati type of differential equations
3.1. The Proof of Main Result Using Lyapunov Functions
It should be recalled that the coefficients of the systems (1.1), (1.7) and of the functionals (1.4), (1.8) have the form if , . In addition to this, we have The formula is useful for the calculation of the particular Lyapunov functions , of the functional (1.8). We get or, the more convenient form Then the system (3.1) has the form Let us assume that for the control system (1.1) the optimal control exists in the form (1.5) independent of the initial value . Regarding the formula (3.6), there exist minimal values of the particular Lyapunov functions , , which are associated with the optimal control. It also follows from the fact that the functions , are particular values of the functional (3.6). Finding the minimal values , by choosing the optimal control is a well-studied problem, for the main results see . It is significant that all matrices , of the integrand in the formula (3.8) are constant matrices, hence, solving the optimization problem they can be considered as matrices of parameters.
Therefore, the problem to find the optimal control (1.5) for the system (1.1) can be transformed to problems to find the optimal control for the deterministic system (3.9), which is equivalent to the system of linear differential equations of type (2.37).
3.2. The Proof of the Main Result Using Lagrange Functions
In this part, we get one more proof of the Theorem 3.1 using the Lagrange function.
We are looking for the optimal control which reaches the minimum of quality criterion Let us introduce the Lagrange function where is the column-vector of Lagrange multipliers. In accordance with Pontryagin's maximum principle, we put the first variations of the functionals , equal to zero and we obtain the system of linear differential equations Then the optimal control can be expressed by The synthesis of the optimal control needs to find the integral manifolds of the solutions of the system (3.12) in the form According to the theory of integral manifolds  we construct the differential matrix equations of the Riccati type for the matrix . Integrating them from time to time and using the initial condition we obtain Lagrange functions for the optimal control We will prove that Differentiating the equality (3.17) with respect to we obtain the matrix equation and extracting the optimal control we obtain differential equation for identical with (3.15). The equality follows from the positive definite matrices , for . Therefore, from (3.17) we get ; moreover, from (3.10) it follows that . Applying the formulas (3.15), (3.16) to the system (3.8) with minimal functionals (3.9), the expression for the optimal control can be found in the form where symmetric matrices satisfy the matrix system of differential equations The systems (3.9), (3.20) define the necessary condition such that the solutions of the systems (1.4) will be optimal. In addition to this, the system (3.8) defines the matrices , of the optimal control in the form We define matrices from the system equations (3.20) in the view of In regards to it can makes the system (3.20) simpler. Then the system (3.20) takes the form (3.3), and formula (3.2) defines the optimal control.
4. Particular Cases
The optimal control for the system (1.1) has some special properties, and the equations determining it are different from those given in the previous section in case the coefficients of the control system (1.1) have special properties or intensities satisfy some relations or some other special conditions are satisfied. Some of these cases will be formulated as corollaries.
Corollary 4.1. Let the control system (1.1) with piecewise constant coefficients have the form
Then the quadratics functional
determines the optimal control in the form
and the matrices satisfy the equations
if , .
The matrices , are the solutions of the systems of the Riccati-type equations:
Remark 4.2. In the corollary we mention piecewise constant coefficients of the control system (4.1). The coefficients of the functional (4.2) will be piecewise as well, but the optimal control is nonstationary.
Corollary 4.3. Assume that Then the optimal control will be piecewise constant.
Taking into consideration that the optimal control is piecewise constant, we find out that the matrices in (4.5) are constant, which implies the form of the system (4.6) is changed to the form The system (4.8) has constant solutions , if conditions (4.7) hold. Moreover, if the random process is a Markov process then the conditions (4.7) have the form and the system (4.8) transforms to the form for which the optimal control is
Corollary 4.4. Let the state of the semi-Markov process be no longer than . Then the system (3.8) has the form Because then
In this case, the search for the matrix in concrete tasks is reduced to integration of the matrix system of differential equations (3.15) on the interval with initial conditions (4.14). In view of , we can expect that every equation (3.15) has a singular point . If has simple zero at the point , then the system (4.6) meets the necessary condition for boundary of matrix in the singular points.
The authors would like to thank the following for their support: the Slovak Research and Development Agency (Project APPV-0700-07), the Grant Agency of the Slovak republic (VEGA1/0090/09), and the National Scholarship Program of the Slovak Republic (SAIA).
- V. M. Artemiev and I. E. Kazakov, Handbook on the Theory of Automatic Control, Nauka, Moscow, Russia, 1987.
- K.G. Valeev and I. A. Dzhalladova, Optimization of Random Process, KNEU, Kyjew, Russia, 2006.
- I. I. Gihman and A. V. Skorohod, Controlable of Random Process, Izdat. “Naukova Dumka”, Kyjew, Russia, 1977.
- I. A. Dzhalladova, Optimization of Stochastic System, KNEU, Kyjew, Russia, 2005.
- V. K. Jasinskiy and E. V. Jasinskiy, Problem of Stability and Stabilization of Dynamic Systems with Finite after Effect, TVIMS, Kyjew, Russia, 2005.
- K. J. Astrom, Introduction to Stochastic Control Theory, vol. 70 of Mathematics in Science and Engineering, Academic Press, New York, NY, USA, 1970.
- R. Balaji, Introduction to Stochastic Finance, University to Conecticut, Academic Press, New York, NY, USA, 1997.
- J. K. Hale and S. M. Verduyn Lunel, Introduction to Functional-Differential Equations, vol. 99 of Applied Mathematical Sciences, Springer, New York, NY, USA, 1993.
- O. Hájek, Control Theory in the Plane, vol. 153 of Lecture Notes in Control and Information Sciences, Springer, Berlin, Germany, 1991.
- X. Liao and P. Yu, Absolute Stability of Nonlinear Control Systems, vol. 25 of Mathematical Modelling: Theory and Applications, Springer, New York, NY, USA, Second edition, 2008.
- G. D. Qushner, Stochastic Stability and Control, Mir, Moscow, Russia, 1969.
- L. Glass and M. C. Mackey, From Clocks to Chaos. The Rythms of Life, Princeton University Press, Princeton, NJ, USA, 1988.
- K.G. Valeev and O. L. Strijak, Methods of Moment Equations, AN USSR, Kyjew, Russia, 1985.
- K. G. Valeev, O. L. Karelova, and V. I. Gorelov, Optimization of a System of Linear Differential Equations with Random Coefficients, RUDN, Moscow, Russia, 1996.
- H. Kwakernaak and R. Sivan, Linear Optimal Control Systems, John Wiley & Sons, New York, NY, USA, 1972.
- K.G. Valeev and G. S. Finin, Constructing of Lyapunov Function, Naukova dumka, Kyjew, Russia, 1981.
- E. A. Barbashyn, Lyapunov Functions, Nauka, Moscow, Russia, 1970.
- R. Bellman, Introduction to Matrix Analysis, McGraw-Hill, New York, NY, USA, 1960.
- K. G. Valeev and O. A. Jautykov, Infinite Systems of Differential Equations, Nauka, Alma-Ata, Russia, 1974.