Abstract
The aim of this paper is to determine the optimal open loop solution and a nonlinear delay-dependent state feedback suboptimal control for a class of nonlinear polynomial time delay systems. The proposed method uses a hybrid of block pulse functions and Legendre polynomials as an orthogonal base for system’s states and input expansion. Hence, the complex dynamic optimization problem is then reduced, with the help of operational properties of the hybrid basis and Kronecker tensor product lemmas, to a nonlinear programming problem that could be solved with available NLP solvers. A practical nonlinear feedback controller gains are deduced with respect to a least square formalism based on the optimal open loop control results. Simulation results show efficiency of the proposed numerical optimal approach.
1. Introduction
Time delays affect systems dynamics in many engineering applications like chemical control systems, biology, and medicine [1, 2]. Delays are also encountered in communication and information technologies like high-speed communication networks [3]. It should be noted that time delay may be, in some applications like communication lines, a source of instability and performance degradation [4]. Time delay system is therefore a very important class of processes whose stabilization [5] and optimization [6, 7] have been of interest to many researchers.
Particularly, many attempts have been made in literature to solve optimal control problems for many classes of linear [8–11] and nonlinear [6, 12, 13] time delay systems. Among them, we recall the application of Pontryagins maximum principle to the optimization of control systems with time delays which was firstly proposed by [14]. It had been shown that it results in a system of coupled two-point boundary-value (TPBV) problem involving both delay and advance terms whose exact solution, except in very special cases, is very difficult to determine (see [15]). Perhaps one of the most effective techniques is dynamic programming approaches (see [2]) for overcoming the complexity of the nonlinear time delay systems in optimal control problems. Of course, application of dynamic programming methods has some difficulties due to the need to provide an appropriate level model and also to define recursive relationships for each case problem. Also a computational algorithm considering a linear approximation of the original system which is defined about a nominal trajectory is offered by [16]. Clearly, using the linear approximation is not reliable and may lead to large errors. Reference [7] proposed an approach based on discretization techniques and necessary conditions to obtain approximate optimal control and the state for optimal control problems with nonlinear delay systems. Despite the good performance of this method, achieving the necessary conditions in some problems and the implementation of approach may be faced with difficulties. So different numerical methods have been proposed to avoid the problems arising from the applications of analytical methods. It is then straightforward that many of the numerical methods dedicated to solving classical optimal control problems have been extended to handle optimal control problems governed by time delay systems.
Typically, direct methods are based on converting the dynamical optimal control problem into static optimization problem. Among direct methods, parametrization technique [17–19] is known to minimize decision variables compared to the discretization of the problem [7]. It is worth noting that parametrization relies basically on orthogonal functions or wavelets [20–22]; however that tool have been used to solve various other problems of dynamic systems like identification (see [8]), tracking control (see [23]), observer based control (see [24]), or minimum time control (see [25]). The main characteristic of this pseudo-spectral technique is that it allows transforming complex dynamic optimization problems to solving a set of algebraic equations in the least square sense in the linear systems case [26, 27] or permits formulating an equivalent nonlinear static programming problem for problems related to nonlinear systems [13, 28, 29].
In recent years, a growing interest has been appeared toward the application of hybrid functions, which is a combination of block pulse and an orthogonal polynomials basis [26]. In the nonlinear time delay optimal control problems context, an approach using hybrid functions which consist of block pulse functions and orthonormal Taylor series (see [15, 29]) had been proposed, where authors propose to solve the necessary and sufficient condition equations for stationary emanating from the Hamiltonian based on state and control coefficients over the basis. Similarly, [28] propose a direct approach based on a hybrid of block pulse functions and Lagrange interpolating polynomials in order to convert the original optimal problem containing multiple delay into a mathematical programming one, where the resulting optimization problem is solved numerically by the Lagrange multipliers method. Reference [27] proposed similar approach based on hybrid functions of block pulse and Bernouilli polynomials, while [30] uses biorthogonal cubic Hermite spline multiwavelets in addition to block pulse functions to constitute the hybrid basis. Although above contributions treat some nonlinear delayed optimal control problem, they do not propose any general nonlinear programming problem that could handle all examples depicted in their works. In fact, for each considered nonlinear system, a nonlinear optimization problem is formulated and then solved with an NLP solver. Furthermore, only open loop control solutions are investigated therein, which is not of great interest in practice.
In the present paper, we introduce a direct method to solve forwardly the finite time quadratic optimal control problem of polynomial systems with delayed state, by the use of hybrid functions of block pulse and Legendre polynomials. The operational matrices of delay and Kronecker product specific to that basis are recalled. At first, the open loop solution of the nonlinear time delay optimal control problem is investigated. Secondly, a suboptimal nonlinear state feedback is determined based on the first part results. Hence, the main contributions in this work could be summarized as follows:
(a) expressing the constraint of the formulated NLP problem properly for the class of polynomial systems; thus the proposed formulation could handle a wide range of nonlinear analytic nonlinear systems. Then, a unified development is carried for that class of systems,
(b) deriving a nonlinear polynomial delay-dependent nonlinear suboptimal state feedback that reproduce the optimal state trajectories determined in the open loop framework,
(c) using an hybrid basis with reduced number of elementary functions, which makes open loop synthesis faster, with a good enough accuracy compared to other approaches, and closed loop solution within a simpler formulation and resolution.
The remainder of the paper is organised as follows. In the second section, hybrid functions and their properties are introduced. In the third section, the open loop numerical solution of the nonlinear time delay optimal control problem is detailed. The suboptimal closed loop framework is presented in the fourth section. In the fifth section, computational results are depicted. Finally, concluding remarks and future works are presented.
2. Hybrid Functions
Hybrid functions , , , have three arguments; and are the order of block pulse functions and Legendre polynomials, respectively, and is the normalized time. They are defined on the interval as [26]
Here, are the well-known Legendre polynomials of order which constitute the base and satisfy the following recursive formula:
We define the vector of block pulse functions , , as follows:
Since is the combination of Legendre polynomials and block pulse functions which are both complete and orthogonal, then the set of hybrid functions is a complete orthogonal system.
In the rest of the paper we notate the dimension of the hybrid basis.
2.1. Operational Matrix of Integration
The integration of can be approximated by [26]
where is the integration operational matrix of order
where
and
2.2. Delay Modeling with Hybrid Functions
A vector function of r dimensional components which are square integrable in can be approximated by a block pulse series as
where .
For an component delay vector variable with
the block pulse series approximation of may be defined as [31]
where
with
and is the number of block pulse functions considered over , and .
Let
Then, it comes [31]
and are called the shift operational matrices, given by
and
It is worth noticing that the Shift operational matrices of hybrid functions could be derived forwardly from those of block pulse functions by
where stands for the Kronecker product.
However, it should be noticed that block pulse functions are fundamental for delay modeling. The choice of depends on and , which issue had been addressed in [26]. In this framework, we propose to choose N as follows:
where is a nonnegative integer, to be chosen bigger than one if possible in order to improve approximation and denotes the nearest integer function [22] (implemented by routine in MATLAB).
2.3. The Integration of the Cross Product
The integration of the cross product of two hybrid functions vectors can be obtained as [26]
where is an matrix. stands for the zero matrix and is an diagonal matrix that is given by
2.4. Kronecker Product Operational Matrix
It would be interesting, for bilinear systems, as it will be proven later, to investigate the Kronecker product operational matrix for hybrid functions. This particular matrix operator derivation, as it is the case for the integration, cross product, and delay operators, is highly inspired of the Kronecker operational matrix of both Legendre polynomials and block pulse functions.
For the block pulse functions, we can state
with
where is the Kronecker product operational matrix of block pulse functions and the matrix is defined in Appendix.
On the other hand, the product of two Shifted Legendre Polynomials and can be expressed by
with
A practical implementation of the latter scalar products is given in [19].
Then, we may write
where is a square matrix.
Then it comes
where is the Kronecker product operational matrix of Legendre polynomials.
Based on relations (26) and (23), we define
where is the Kronecker product operational matrix of hybrid functions.
3. Numerical Solution of the Nonlinear Time Delay Optimal Control Problem
3.1. Description of the Studied System
We consider the nonlinear continuous system which can be represented by the following state space representation:
where is the state vector, is the delayed state where denotes the time delay, is the control vector, and from into and from into are nonlinear analytic functions of and .
Note that, any functions and could be approached using truncated series of the Kronecker power of and as follows:
where , and are constant matrices. denotes the Kronecker power of the state vector (see Appendix).
And
where , for , are defined by
where , , and are constant matrices.
Notice that is composed of three terms, the first is a function of , the second depends on , and the third is a function of . Consider the first term of ; we note it . It could be written as follows:
Then could be generalized to the following expression:
3.2. Statement of the Problem
Consider the system defined by (28), (29), and (33) with an initial condition . Our objective is firstly to find the optimal open loop control , which minimizes the performance index:
where and are positive semidefinite matrices and R is symmetric positive definite with appropriate dimensions.
The direct approach presented in this paper is based on expanding system equations (28), (29), and (33) as well as objective function (34) to be minimized over an hybrid functions basis. Hence, the main purpose is to transform the optimal control problem under dynamic constraints to a nonlinear programming problem. To this end, each of the state and control variables is approximated by a finite length of unknown parameters as follows:
where and are unknown state and control parameters, respectively. Applying the operator (see Appendix) and related Kronecker product property [32] yields
where and are and identity matrices.
Moreover, at the initial time, , the initial state could be written
where
is an constant vector.
For clarity purpose, let us denote as the whole unknown parameters vector. and are, respectively, the state parameters and the control ones, such that
and .
According to (14) and (17), the delayed state coefficients are given by
3.3. Optimal Control Problem Reformulation Using Hybrid Functions
The cost function (33) is composed of two parts. The first is the terminal penalty of the state, while the second is known to be the running cost.
3.3.1. Cost of the Final State Approximation
At the final time, , the state approximation could be written
It is important to mention here that hybrid functions inherit an important property from Legendre polynomials(). In fact, the subset verifies
The rest of hybrid functions are null at .
The cross product of two hybrid functions at the final time is given by
where is matrix. stands for the all-ones matrix.
Hence, the terminal penalty of the state could be approximated as follows:
3.3.2. Cost of the State Trajectory Approximation
The integral term in the criterion is approached now as
which is equivalent to
Using the integral of the cross operational matrix , it reduces to
3.3.3. System Path Approximation
The expansion of the system state over a hybrid basis requires the development of functions and over that basis. To this end, several preliminary lemmas need to be introduced.
Lemma 1. The development of the Kronecker power of the state vector over a hybrid basis gives wherewithandWe recall that denotes the Kronecker power of the state coefficients , with being the operational matrix of the Kronecker product.
Proof. The proof of this lemma needs only a few manipulations.
Notice that results of Lemma 1 could be applied to the Kronecker power of the delayed state coefficients (i.e., and express it in terms of decision variable by the mean of relation (32):
Expansion of over the Hybrid Basis. The third term of (29) could be developed over the hybrid basis as follows:
where we notate .
Now, could be approached as follows:
where
Expansion of over the Hybrid Basis. Notice that the first term of under the sum could be expanded over the hybrid basis as follows:
while the second one could be derived similarly.
The third term could be approached as
where and belong to . Then it comes
with
Expansion of System Equation over the Hybrid Basis. The integration of the system equation by introducing the operational matrix of integration with respect to notations (58) and (54) gives
Our objective is to express the constraint (60) in terms of decision variables and ; to this end we apply the operator to (60). That allows us to state
Using the linearity property of the operator it comes
which is equivalent to
Lemma 2. where the matrix is defined in Appendix.
Proof. The proof of this lemma needs only a few manipulations.
Notice that results of Lemma 2 could be applied to the Kronecker power of the delayed state coefficients (i.e., ) and express it in terms of decision variable by the mean of relation (42). We note
Applying the operator to yields
Similarly
Finally, the system path constraint could be implemented using the following equation:
Now, it could be noticed that the system path constraint is expressed properly in terms of unknown parameters and .
3.4. The Nonlinear Programming Problem
The optimal control problem has been approximated by a nonlinear programming problem and is given by the following: find the optimal vector of the unknown parameters and that minimizes
subject to (68).
One has
The mathematical programming problem can be solved by using available nonlinear programming solvers like IPOPT or the routine of the MATLAB Toolbox.
After solving the latter nonlinear programming problem and determining the optimal value of the unknown parameters vector , these parameters are substituted back into (28) to determine the optimal state vector and the optimal control.
4. Suboptimal Feedback Control
Once the optimal open loop results are obtained by solving the nonlinear programming problem given by (69)-(68), let us note
the optimal state and control coefficients.
We are interested now, based on previous results, to synthesize the following nonlinear state feedback control law:
The idea is to find control matrices and such that the optimal vector (71) verifies the control equation (72).
Expanding (72) over the hybrid basis yields
Substituting the control and state coefficients with their optimal values and applying the operator give
with and .
Finding control parameters could be then reduced to solving, in the least square sense, the following problem:
where
5. Computational Results
5.1. Example 1
Consider the system [33]
When , the above system has two equilibria, one which is at the origin and the other one at .
In this example, we aim to minimize the following criterion in order to find optimal states and open loop control, and , then a suboptimal control is characterized.
Nonlinear system (77) could be written under a polynomial form (28), (29), and (32) with
The time delay is considered as .
5.1.1. Open Loop Study
The development presented above is implemented in this subsection by using both hybrid functions (HFs) and block pulse functions (BPFs).
Table 1 summarizes considered parameters for simulations below and obtained performances indexes with the different bases. It is then clear that hybrid basis is superior over the piecewise constant one, both with the same number of elementary functions.
Simulation results for the above open loop controlled system initialized with are given in Figures 1 and 2 based on hybrid and block pulse functions.
5.1.2. Closed Loop Framework
The state feedback control gains designed based on hybrid and block pulse open loop frameworks are as follows:
Controlled state trajectories, obtained with determined gains, are depicted in Figure 3. It is shown that the hybrid functions technique is also better in closed loop.
Figure 4 shows the optimal states trajectories obtained by minimizing the formulated NLP problem, by using the hybrid of block pulse and Legendre polynomials basis, over a finite horizon . Controlled states with obtained suboptimal feedback are drawn on the same figure over a simulation time . It could be seen that system states converge to the origin equilibrium with respect to imposed criterion.
Figure 5 exposes optimal control and suboptimal state feedback control signals using HFs.
5.2. Example 2: A Two-Stage Chemical Reactor
In this section, we consider a cascade chemical system with two reactors [34]
where , are the compositions, is a known time delay, are the recycle flow rates, are the reactor residence times, are the reaction constants, is the feed rate, are reactor volumes, and are nonlinear functions for describing the system uncertainties and external disturbances. Note that (80) is the transformed reactor model given by [34]. The original one may have a nonzero equilibrium point and the compositions actually denote the deviations from the equilibrium point.
Let , , , , and . The uncertainties are the functions and , respectively, with .
The simulation is done by taking for and results are given in Figures 3 and 4.
We consider in this example the criterion in order to find optimal states and open loop control, and , then suboptimal control gains such that are investigated.
Nonlinear system (80) could be written under a polynomial form (28), (29), and (32) with
Table 2 gives simulations parameters and obtained Performances indexes with the two bases utilized in Example 1. Also, state feedback gains are included.
It is worth noting that for this particular system, control gains determined by the mean of block pulse functions are not stabilizing. While hybrid functions results are illustrated on Figures 6 and 7, controlled states with obtained suboptimal feedback designed over a finite horizon are drawn on the same figure over a simulation time . It could be seen that suboptimal system states coincide perfectly with optimal solution.
Figure 7 exposes optimal control and suggested suboptimal nonlinear state feedback control signals. The proposed nonlinear feedback reproduces sharply the optimal open loop control.
6. Conclusion
In this paper a practical approach is developed to solve the problem of finite time quadratic optimal control for polynomial time delay systems. The proposed method is based on the expansion of the system model on a complete set of orthogonal hybrid of block pulse and Legendre polynomials. Two types of optimal control laws have been investigated. In the first step, the method focuses on the determination of the open loop optimal control law. Thus, by defining a general NLP problem for the considered system class, in the second step, a nonlinear delay-depending state feedback control law has been derived in order to meet the optimal states trajectories. The developed results have been illustrated on different examples of nonlinear time delay systems; namely, a two-stage chemical reactor and the obtained results are significant. Note that the proposed method may be enhanced, by decreasing the polynomial system matrices order when introducing the nonredundant form. In addition, if the state information is not completely available, the output feedback is a good choice. The method is feasible for the output feedback control and could be even extended to handle dynamic nonlinear state/output feedback synthesis. Moreover, the presented development is limited to some class of analytical nonlinear systems. In future work, we intend to apply the presented method to the class of switched systems, where the used hybrid functions seems to be convenient to treat that models which are a mixture of smooth functions and piecewise constant signals.
Appendix
A.
A.1. Kronecker Product and Function Properties
The Kronecker power of order , of the vector is defined by
For any matrices X, Y, and Z having appropriate dimensions, the following property of the Kronecker product is given [32]:
where denotes the vectorization operator of a matrix [32].
Letting ,,, and matrices with appropriate dimensions, we recall the following properties [32]:
A.2. Function
An important matrix-valued linear function of a vector, denoted as , is defined as follows.
If is a vector of dimension , then is the matrix verifying
A.3. Definition
We note , the dimensional unit vector which has in the element and zeros elsewhere. The elementary matrix of dimension could be defined by [32]
It has on the element of coordinates and zeros elsewhere.
Let and ; we have
where
The matrix , with respect to dimensions, could be also used as follows:
where
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.