Abstract
This paper revisits the problem of synthesizing the optimal control law for linear systems with a quadratic cost. For this problem, traditionally, the state feedback gain matrix of the optimal controller is computed by solving the Riccati equation, which is primarily obtained using calculus of variations- (CoV-) based and Hamilton–Jacobi–Bellman (HJB) equation-based approaches. To obtain the Riccati equation, these approaches require some assumptions in the solution procedure; that is, the former approach requires the notion of costates and then their relationship with states is exploited to obtain the closed-form expression of the optimal control law, while the latter requires a priori knowledge regarding the optimal cost function. In this paper, we propose a novel method for computing linear quadratic optimal control laws by using the global optimal control framework introduced by V. F. Krotov. As shall be illustrated in this article, this framework does not require the notion of costates and any a priori information regarding the optimal cost function. Nevertheless, using this framework, the optimal control problem gets translated to a nonconvex optimization problem. The novelty of the proposed method lies in transforming this nonconvex optimization problem into a convex problem. The convexity imposition results in a linear matrix inequality (LMI), whose analysis is reported in this work. Furthermore, this LMI reduces to the Riccati equation upon imposing optimality requirements. The insights along with the future directions of the work are presented and gathered at appropriate locations in this article. Finally, numerical results are provided to demonstrate the proposed methodology.
1. Introduction
Optimal control is a heavily explored and still developing area of control engineering where the objective is to design a control law so as to optimize (maximize or minimize) a given performance index (cost functional) while driving the states of a dynamical system to zero (regulation problem) or to make output track a reference trajectory (tracking problem) [1]. The generic optimal control problem (GOCP) is given as follows: notation: throughout this article, lowercase alphabets represent scalar quantities, lowercase bold alphabets represent vector quantities, and uppercase alphabets represent matrices, and also, the quantities with “star” in the superscript correspond to the optimal trajectory.
The GOCP computes an optimal control law which minimizes the performance index/cost functionalsubject to the system dynamics to give the desired optimal trajectory . Here, is the running cost, is the terminal cost, is the state vector, and is the control input vector to be designed. Also, and are continuous.
Since, the aforementioned problem corresponds to optimization of the cost functional subject to dynamics of the system considered and possibly constraints on input(s) and/or state(s), the calculus of variations (CoV) is generally employed to address optimal control design problems [1, 2]. The assumption of existence of optimal control is usually the first step while using CoV techniques. Subsequently, the conditions which must be satisfied by such an optimal control law are derived. Hence, only necessary conditions are found, and sufficiency of these conditions is not guaranteed. Furthermore, the obtained control law is usually only locally optimum. Nevertheless, there are results available in the literature which provide restrictions under which the necessary conditions indeed become sufficient and the global optimal control law is obtained [3–5]. Note that, in solving optimal control design problems, the CoV-based methods use the notion of so-called costates (which are not actually present in the system). Moreover, in the solution procedure, the existence of a linear relationship between the states and the costates is exploited to compute the closed-form expression of the optimal control law (this is particularly true for linear quadratic problems) (see [6] for more details).
Alongside CoV, another tool, namely, dynamic programming (DP) (introduced by Bellman), has also been explored to solve optimal control problems. The application of DP to optimal control design problems for continuous linear systems leads to the celebrated Hamilton–Jacobi–Bellman (HJB) equation which also gives a necessary condition for optimality [1]. Nevertheless, this equation also provides sufficiency under the following mild conditions [2, 7]: there exists a continuously differentiable optimal cost function and the gradient of optimal cost function with respect to the state vector equals the costate which corresponds to the optimal trajectory. For example, consider the optimal control design problem for the system [8] with the performance measure as . For this problem, the optimal cost function is , and hence, the HJB equation is not defined at because of nondifferentiability of . From the aforementioned observations, a solution method for optimal control design which does not require these conditions is desirable. As shall be demonstrated in this article, the Krotov solution methodology is indeed such a methodology. In fact, this methodology provides sufficient conditions for existence of the global optimal control law without using the notion of costates and any a priori information regarding the optimal cost function [2].
Starting from the sixties, the results on sufficient conditions for the global optimum of the optimal control problem were published by Krotov [9, 10]. These conditions have been derived from the so-called extension principle [11]. The first step, while employing these conditions, is a total decomposition of the OCP with respect to time via an appropriate selection of the so-called Krotov function [2, 12, 13]. Once such a decomposition is obtained, the problem is reduced to a family of independent elementary optimization problems parameterized in time t. It has been shown in [2] that the two problems—the original OCP and the optimization problem resulting from decomposition—are completely equivalent. The method, however, is abstract in the sense that the selection of the Krotov function is not straightforward and is very problem specific [11]. A number of works have used the Krotov methodology for solving OCPs encountered in control of structural vibration problems in buildings [12], such as MEMS-based energy harvesting problem [13], magnetic resonance systems [14], quantum control [15], and computation of extremal space trajectories [16, 17]. However, the equivalent optimization problems in all these articles are nonconvex, and hence, the iterative methods, one of them being the Krotov method, are employed to obtain their solutions. To address this issue, we propose a novel method to directly (noniteratively) synthesize optimal controllers for linear systems using Krotov sufficient conditions. The innovation in our approach lies in transforming the nonconvex optimization problem into a convex optimization problem by a proper selection of Krotov functions. Upon convexity imposition, for the linear quadratic regulation (LQR) problem, a matrix inequality is obtained which can be easily converted to a linear matrix inequality (LMI) using Schur complement. Following the argument “However, it is much less appreciated how the other relations (LMIs) enter in to the theory” in [18] (regarding the LMIs whose derivation and analysis are less appreciated as compared to those of the famous Riccati equation), this work demonstrates how this LMI enters into the theory. Specifically, it results in the convexity imposition on the equivalent optimization problem (which is obtained by decomposition of the original optimal control problem using the Krotov function). Moreover, analysis of the obtained LMI is also reported in this article. In [19], the methodology was demonstrated for finite horizon linear quadratic optimal control problems, and the Krotov function was taken to be a positive definite quadratic function. This article presents a rather detailed discussion of the results along with extension of the methodology to infinite horizon problems. It differs from the study [19] in that the Krotov function here is taken as a quadratic function; neither symmetry nor positive definiteness is imposed upon this function. Moreover, the analysis of the LMI which results from convexity imposition is also presented. Considering the aforementioned points and given the fact that the Krotov framework remains highly unexplored in the literature (to the best of authors’ knowledge), this article may serve as a background for further exploration of this framework to more involved control problems, viz., nonlinear optimal control problems, distributed optimal control problems, etc. In summary, the contribution of this work is as follows: it exhaustively describes the methodology for solving the standard linear quadratic optimal control problems (both finite and infinite horizon) using Krotov sufficient conditions, it solves the equivalent optimization problems via convexity imposition: a technique which is not used in the previous work which uses Krotov conditions and then the analysis of resulting LMI is presented, and it provides the insights which result upon synthesizing the optimal control laws and may also lay the foundation upon which the Krotov sufficient conditions may be employed for solving more complex optimal control problems, viz., nonlinear optimal control problems.
The rest of this article is organized as follows: In Section 2, the preliminaries of linear quadratic optimal control problems are discussed and the solution methodologies based on the calculus of variations- (CoV-) based method and Hamilton–Jacobi–Bellman (HJB) equation-based method are outlined. The assumptions as encountered in these approaches are also discussed. In Section 3, the background literature of Krotov sufficient conditions and their application to the problems considered are detailed. This section also discusses a number of insights which result while solving the considered optimal control problems. These insights are gathered as remarks at appropriate locations in this section. In Section 4, the analysis of LMI obtained upon convexity imposition is presented. The Krotov iterative method is also discussed in brief in this section. In Section 5, the proposed method is demonstrated through numerical examples. Finally, the concluding remarks and future scope of the work are presented in Section 6.
2. Preliminaries and Problem Formulation
In this section, solution procedures of linear quadratic regulation (LQR) and linear quadratic tracking (LQT) problems using CoV- and HJB equation-based approaches are briefly discussed in order to highlight the assumptions used in these approaches.
2.1. OCP 1 (Finite Horizon LQR Problem)
Compute an optimal control law which minimizes the quadratic performance index/cost functionalsubject to the system dynamics and drives the states of the system to zero (regulation). Here, is given, is free, and is fixed. Also,
The solution using the CoV technique comprises the following four major steps:(i)Formulation of Hamiltonian function: the Hamiltonian for the considered problem is given as where is the costate vector.(ii)Obtaining the optimal control law using the first-order necessary condition: the optimal control law is obtained as(iii)Use of state and costate dynamics and a transformation to connect states and costates for all : the boundary conditions (i.e., being fixed and being free) lead to the following boundary condition on : . Then, the transformation to connect costate and state vectors is used to compute the optimal control law as(iv)Obtaining the matrix differential Riccati equation: finally, taking the derivative of equation (5) and substituting the state and costate relationship, the following matrix differential Riccati equation (MDRE) is obtained which must satisfy for all :
Furthermore, the solution using the HJB equation requireswhere is the optimal cost function and is the optimal control law. To solve (8), the boundary condition is used with assumed to bewhere is a real, symmetric, positive-definite matrix to be determined. Substituting (9) into (8), we get
This equation is valid for any if
Finally, , and thus, the solution is the same as that obtained using CoV. Summarizing the above, the global optimal control law is given bywhere is the solution ofwith the boundary condition .
2.2. OCP 2 (Finite Horizon LQT Problem)
Compute an optimal control law which minimizes the following quadratic performance index/cost functional:where subject to the system dynamicssuch that the output tracks the desired reference trajectory . Here, is the error vector, is given, is free, and is fixed. Also,
Similarly to the solution of LQR, the CoV- and HJB equation-based approaches yield the optimal control law aswhere and satisfywith boundary conditions and , respectively. Note that, in the HJB approach, the optimal cost function has to be guessed.
Although the CoV- and HJB equation-based approaches as described above are widely employed for solving OCPs, there are some assumptions associated with these approaches in their solution procedure. Specifically, the CoV-based approach uses the notion of costates and their relationship with states for all time (5) to compute the optimal control law. Similarly, the HJB equation-based approach requires the existence of the continuously differentiable optimal cost function, and its gradient with respect to the state is the costate corresponding to the optimal trajectory [7]. Thus, the information about the optimal cost function must be known a priori. The angle of our attack is to synthesize an optimal control law using Krotov sufficient conditions, where the above issues are not encountered in the solution procedure. However, it is well known that the control law using these conditions is synthesized through an iterative procedure. The main nontrivial issue to be tackled is how to obtain noniterative solutions of optimal control problems using Krotov conditions. The next section answers this question for linear quadratic optimal control problems.
3. Computation of Optimal Control Laws
In this section, solutions of the LQR and LQT problems using Krotov sufficient conditions are detailed.
3.1. Krotov Sufficient Conditions in Optimal Control
The underlying idea behind Krotov sufficient conditions for global optimality of control processes is the total decomposition of the original OCP with respect to time using the so-called extension principle [2].
3.1.1. Extension Principle
The essence of the extension principle is to replace the original optimization problem with complex relations and/or constraints with a simpler one such that they are excluded in the new problem definition but the solution of the new problem still satisfies the discarded relations [11].
Consider a scalar-valued functional defined over a set (i.e., ) and the optimization problem as follows.
Problem 1. Find such that where . Instead of solving Problem 1, another equivalent optimization problem is solved. Let L denote the equivalent representation of the original cost functional. Then, a new problem is formulated over , a superset of , as follows. Equivalent Problem 1. Find such that where .The equivalent problem is also called the extension of the original problem. The method of choosing of the equivalent functional L is not unique, and the selection is generally made according to specifications of the problem under consideration. This freedom in the selection of the equivalent functional can be exploited to tackle the generic nonconvex optimization problems. Also, it is necessary to ensure that , so that the optimizer is actually the optimizer of original Problem 1 [2]. The idea behind the extension principle is illustrated in Figure 1. Clearly, the application of the extension principle requires appropriate selection of the equivalent functional L and the set .
(a)
(b)
3.1.2. Application of Extension Principle to Optimal Control Problems
The equivalent problem of the GOCP which is obtained by the application of the extension principle is given in the following theorem which provides a concrete definition of the equivalent functional L and the set .
Theorem 1. For the GOCP, let be a continuously differentiable function. Then, there is an equivalent representation of (1) given aswhere
Proof. See Section 2.3 in [2] for the proof.
The equivalent functional leads to a sufficient condition for the global optimality of an admissible process ( pair which satisfies the dynamical equation and the input/state constraints).
Theorem 2. (Krotov sufficient conditions). If is an admissible process such thatthen is an optimal process. Here, is the terminal set for admissible of ; that is, if is admissible, then .
Proof. See Section 2.3 in [2] for the proof.
Remarks. Some remarks which follow from Theorem 1 and Theorem 2 are now briefed as follows:(1)The functional is the equivalent functional of the original functional J in (1). Specifically, .(2)The function , known as the Krotov function, can be any continuously differentiable function, and each leads to a different equivalent functional . A general way to choose this function is not known, and the selection is usually done according to the specifics of the problem in hand. Moreover, it is possible that the same solution results for different selections of [2]. While this ad hocness in choosing the Krotov function may seem burdensome, it provides enough freedom in making the selection according to the specifications of the problem in hand [11, 20]. In essence, this work exploits the ad hocness in selecting the Krotov function.(3)The original GOCP is transformed to an equivalent optimization problem of the functions s and over the sets , and . Thus, the dynamical equation constraint is excluded from the optimization problem by defining .(4)If the optimization problem formulated in Theorem 2 is feasible (i.e., an admissible process satisfying the sufficient conditions in Theorem 2 can be found), then the function is called the solving function.(5)Clearly, different selections of result in different optimization problems in Theorem 2, and thus, the selection of is crucial to effectively solve the optimization problems in Theorem 2. Furthermore, the necessary conditions for existence of the optima (optimum) of the functions s and coincide with the popular Pontryagin’s minimum principle [20]. Moreover, a specific setting of the Krotov function leads to the popular Hamilton–Jacobi–Bellman equation [2, 20]. These two observations lead to the conclusion that Krotov sufficient conditions are in fact the most general sufficient conditions for global results in optimal control theory.The solution to the equivalent optimization problem given in Theorem 2 is generally computed using sequential methods because the resulting optimization problem is nonconvex. Such a sequence of processes is called an optimizing sequence [10]. One such iterative method is the Krotov method in which an improving function is chosen at each iteration. To avoid iterative methods, we convexify the equivalent problems in Theorem 2 via a suitable selection of the Krotov function. The idea is illustrated in Figure 1 where the set —the set of solving functions which result in the convexity of equivalent optimization problems in Theorem 2—is the set of interest. In the next section, the suitable Krotov functions for LQR and LQT problems are proposed. In the following, we use as the cost functional instead of , and the time variable t is dropped wherever it is required for the sake of simplicity.
3.2. Solution of OCP 1 (LQR Problem)
The equivalent optimization problem for OCP 1, as per Theorem 2, is given as follows. Equivalent OCP 1: compute an admissible pair which is as follows:(i), where(ii), where
The next proposition is one of the main results of this paper, where we propose a suitable Krotov function which will be useful in computing a direct solution to OCP 1.
Proposition 1. For Equivalent OCP 1, let the Krotov function be chosen asand every element of the matrix P is differentiable . Then, the following statements are equivalent:(a) and are convex functions in and , respectively.(b)P satisfies the following matrix inequalities:(i)(ii)
Proof. With q selected as in (23), the function becomesAdding and subtracting the term , we getSince , there exists a unique positive definite matrix [21], say , such that and . Now rearranging the terms in (25), we getClearly the second term in (26) is strictly convex. Now, s is convex iff the following condition is satisfied:Moreover, with q as in (23), is given asFinally, is convex iff
Corollary 1. With selected as in Proposition 1, the following statements are true:(1)The function is a solving function for OCP 1.(2)The global optimal control law for OCP 1 is given bywhere P is the solution of the matrix differential equationwith the final value .
Proof. Clearly, if P satisfies (27), then s is independent of and the obtained control law results in an admissible process. Hence, the selected q is the solving function. Furthermore, for minimization, it is required that the second term in (26) is zero which gives the optimal control law as
3.2.1. Solution of OCP 1 with Final Time (Infinite Horizon LQR)
Next, the infinite-final time LQR problem for a linear time invariant (LTI) is considered. For this case, the terminal state weighing matrix . The problem statement now is as follows.
Problem 2. (infinite horizon LQR problem). Compute an optimal control law which minimizes the quadratic performance index/cost functionalsubject to the system dynamics and drives the states of the system to zero (regulation). Here, is given, and is free. Also, . The quantities A, B, Q, and R are time-independent.
Solution. For this problem, the matrix differential equation (27) needs to be solved with the boundary condition . Computing this solution is equivalent to solving the following algebraic equation:and the resulting optimal control law is given asSimilar to the finite-final time case, it is easy to verify that the function (where P is a time-invariant (constant) matrix) is a solving function. Note that since the final time , it is necessary to ensure the stability of the closed-loop system. The next lemma provides a proof of the closed-loop stability under another condition on the P matrix.
Lemma 1. For the system , the control input (35) ensures the stability of the closed loop if P satisfies (34) with .
Proof. Let the Lyapunov function be . Then,It is easy to verify the quantity is negative semidefinite if and negative definite if . Thus, using Lyapunov theory [22], the closed-loop system is stable if and asymptotically stable if .
Remarks. The matrix differential equation (31) reduces to the popular matrix differential Riccati equation (MDRE) for a symmetric P matrix. Also, the algebraic equation (34) admits more numbers of solutions than the MDRE as demonstrated in Example 3 in Section 5. A more rigorous analysis and application of these solutions are the subject of future research.
3.3. Solution of OCP 2 (LQT Problem)
The equivalent optimization problem for OCP 2 is given as follows. Equivalent OCP 2: compute an optimal control law which is as follows:(i), , where(ii), where
Another main result of this paper is given in the next proposition, which will be useful in computing the direct solution to OCP 2.
Proposition 2. For Equivalent OCP 2, let the Krotov function be chosen asand every element of the matrix P and the vector is differentiable . Then, the following statements are equivalent:(a) and are bounded below and convex in and , respectively.(b)(i) P satisfies the following matrix inequalities:(1)(2)(ii) satisfies the vector differential equation with the boundary condition
Proof. With q chosen as in (39), the function is given asAdding and subtracting the terms , , and , we getSince , there exists a unique positive definite matrix [21], say , such that . Now rearranging the terms in (43), we getThe second term in (44) is positive definite and strictly convex. The third term is linear in , and the fourth term is independent of and . The first term in (44) is quadratic in , and it is convex iff it is positive semidefinite. Hence, the function is convex iffSince the second term in (44) is linear in , it is bounded below ifFinally, the term independent of and , i.e., , is bounded because and are bounded [23]. Next, with as in (39), is given asSimilar to the case of s, is convex iffFinally, is bounded below ifThe following corollary computes the optimal control law for OCP 2.
Corollary 2. With selected as in Proposition 1, the following statements are true:(1)The function is a solving function for OCP 2.(2)The global optimal control law for OCP 2 is given bywhere P is the solution of the matrix differential equationwith the final value , and satisfies the differential equation with the final value .
Proof. Clearly, if P satisfies (50) and satisfies (45), then s is independent of and the obtained control law yields an admissible process. Hence, is the solving function. The third term in (44) is strictly convex and attains a minimum value when
3.3.1. Solution of OCP 2 with Final Time (Infinite Horizon LQT)
Next, the infinite-final time LQT problem for a linear time invariant (LTI) is considered. For this case, the terminal cost . The problem statement now reads as follows.
Problem 3. (infinite horizon LQT problem). Compute an optimal control law which minimizes the quadratic performance index/cost functionalsubject to the system dynamics and and drives the states of the system to the desired trajectory . Here, is the error vector, is given, is free, is fixed, and is free. Also,
Solution. For this problem, similar to the case of regulation, the matrix differential equation (50) needs to be solved with the boundary condition . Similarly, the differential equation (45) needs to be solved with the boundary condition . Computing these solutions is equivalent to solvingand the resulting optimal control law is given asSimilar to the finite-final time case, it is easy to verify that the function (where P is a time-invariant (constant) matrix) is a solving function. In this case too, similar to the case of regulation, the stability of the closed loop is ensured if .
Remarks. As is clear from the solution methodologies proposed above, the solution of the considered problems does not require the notion of costates and any a priori information regarding the optimal cost function. Rather, the whole solution technique depends upon the selection of the Krotov function which directly affects the nature of equivalent optimization problems.
4. Analysis of the LMI Obtained upon Convexity Imposition
As mentioned earlier in Introduction, it was asserted in [18] that the origin of Riccati-type linear matrix inequalities (LMIs) is not much appreciated in the theory. This section tries to address this issue. Specifically, note that the convexity imposition for EOCP 1 for the infinite horizon case (in Section 3.2) results in the following matrix inequality:
Thus, this inequality preserves the strict convexity of EOCP 1 for the infinite horizon case. Furthermore, for a symmetric P, this inequality reduces to
Using Schur’s complement [24], (56) can be written as the following linear matrix inequality (LMI):
Thus, (56), which is essentially Riccati LMI, preserves the convexity of EOCP 1. This complete explanation for the origin of (57) fills the gap in the literature (regarding the origin of this LMI as stated in [18]).
Next, an analysis of this inequality is presented. Specifically, the following question is being addressed: Is it possible to solve (56) to obtain stabilizing (and of course suboptimal) control laws (of the form ) for the LQR problem? The motivation behind this query is the fact that once a stabilizing solution using (56) is found, it is straightforward to find how suboptimal the obtained solution is with respect to the optimal solution obtained using equation (27). This is true because it is known that if P satisfies (56), then the equivalent optimization problem is convex, and thus, it is straightforward to find how far is the obtained suboptimal solution from the global optimum. Thus, the problem of suboptimal control of linear systems with quadratic cost is also addressed upon solving (56). Note, moreover, that another LMI-based approach for the suboptimal control of linear systems with quadratic cost was utilized in [25] to design suboptimal distributed control laws for multiagent systems. With this motivation, the analysis of (56) is presented next. Specifically, sufficient conditions for the existence of a matrix, say, , which satisfy this inequality and result in the stabilizing control laws of the form are derived in the next proposition.
Proposition 3. Define(i) as the solution of the algebraic Riccati equation, i.e., , (ii), If positive constants α, δ, and γ and a positive definite matrix such that(1) (note that this condition implies that the eigenvalues of A lie to the left of the line in the complex plane)(2) with (3) then the following hold:(i)(ii) is Hurwitz, where
Proof. Since , it follows that because . ConsiderThis completes the proof of part .
Again, implies that because . Next, considerThis completes the proof of part .
Proposition 3 provides a result for suboptimal linear quadratic optimal control design. Further exploration and application of this result are the topic of future research.
4.1. Krotov Method
The Krotov method is one of the iterative methods to solve the equivalent optimization problems of Theorem 2. The steps of this method are as follows:(1)Choose any admissible control and compute the corresponding admissible process using the dynamical equation of the system. Also, compute the cost .(2)Construct a function such that(a)(b)(3)Construct the control input such that(4)Compute the cost of the resulting admissible process and compare it with the cost obtained in step 1 to determine the extent of improvement achieved, viz., compare the difference between the two costs with a predefined tolerance, say ϵ. If the difference is less than ϵ, go back to step 2 with the improved process obtained in step 3; otherwise, stop.
This method is demonstrated in Figure 2. Now, the optimal control problem for a scalar system is solved to clarify the approach of the Krotov method.
Problem 4. For the system with , compute the optimal control law so as to minimize .
Solution. (i)Choose an admissible process as which gives . The corresponding cost is computed to be .(ii)Next, a function ϕ is chosen so as to maximize over x. Upon a quadratic selection of ϕ as , the function s is given as and . Choosing p such that and clearly turns out to be appropriate selection for this case. Then, the improved process is given as where and . The corresponding cost is computed to be .(iii)Next, the improved function is again selected to be a quadratic function , and the corresponding equation for p is given aswith . The corresponding improved process is given by with , and the corresponding cost is computed as . It was found that, upon further iterations, no improvement of cost was achieved, leading to the conclusion that this process is the globally optimal process.
5. Numerical Examples
In this section, the proposed solution methodology is demonstrated via numerical examples. Firstly, the finite-time optimal control laws for a scalar linear time-varying system are computed. Then, infinite-time optimal control laws for a linear time-invariant (LTI) multi-input multioutput (MIMO) system are computed.
Example 1. Compute the optimal control law for the system which minimizes the performance index with and .
Solution. Following Theorem 2, the equivalent optimization problem to be solved is given aswhere and According to Proposition 1, the Krotov function is chosen asand then the functions s and are read asClearly, the function s is nonconvex. Because of this, the equivalent problems (62) and (63) are solved using an iterative method, such as the Krotov method in which q is chosen appropriately at each iteration. Instead, using Proposition 1, the direct solution can be computed by choosing such thatFinally, using Corollary 1, is chosen such that and . The solution of the above differential equation is given asand the optimal control law is computed asThe plots of and the state response of the closed-loop system and the optimal control input are shown in Figures 3(a) and 3(b), respectively.
(a)
(b)
Example 2. Compute the optimal control law for the system given as which minimizes the performance index with , the reference trajectory , , and .
Solution. The optimization problem to be solved is given asLet the Krotov function be chosen asand then the functions s and are given asClearly, the functions s and are nonconvex. Again, using Proposition 2, the direct solution can be computed by choosing and such thatFinally, using Corollary 1, p is chosen such that , and . The obtained and are as shown in Figure 4.
Finally, the optimal control law is computed asThe closed-loop response is as shown in Figure 4.
Next, infinite horizon optimal control problems for the LTI MIMO system are considered.
(a)
(b)
Example 3. For the MIMO system , compute an optimal control law to minimize the performance index where , , , and . Also, .
Solution. The optimization problem to be solved is given aswhereThis function is nonconvex. However, as shall be demonstrated, the function can be convexified if is chosen as per Proposition 1.
Let the Krotov function beThen, the function is given asNext, using Proposition 1 and Corollary 1, the direct solution can be obtained by choosing P so as to satisfyThis selection results in a convex s. This fact is illustrated by the plots in Figure 5 where s is plotted against and for and along the lines and with .
Equation (78) admits the four solutions , , , and
Finally, to ensure the stability of the closed loop, P is chosen such that Lemma 1 is satisfied, i.e., . It can be easily verified that only satisfies this requirement. Thus, is the required value of the matrix P in this case. Finally, the optimal control law is given asThe closed-loop response is shown in Figure 6(a).
(a)
(b)
(a)
(b)
Example 4. For the MIMO system and , compute an optimal control law to minimize the performance index where , , , , is the error defined as , and is the reference defined as with . Also, .
Solution. The optimization problem to be solved is given aswhereWe use the Krotov function asThen, the function is given byAgain, it is easily verified that the function is indeed convex for the selection of P and as in Corollary 2. P is computed using (78) similar to the case of regulation, and is a time-varying function which can be computed as the steady-state solution as in (53):The convexity of s by these selections of P and is illustrated by the plots in Figure 5 for similar to the case of regulation.
The P matrix is chosen to be (similar to the case of regulation) , and is calculated to beThe plots of and are shown in Figure 7 which clearly show their boundedness. Finally, the optimal control law is calculated using Corollary 2, and the closed-loop response is given in Figure 6(b).
6. Conclusion
In this paper, we propose a novel method to compute globally optimal control laws for linear quadratic regulation and tracking problems based on Krotov sufficient conditions. The solution to the linear optimal control problem has been widely addressed in the literature using the celebrated CoV/HJB methods. These methods synthesize the global optimal control law, which is unique and requires some forced assumptions. In order to address this issue, we solved the optimal control problem using Krotov sufficient conditions, which do not require the notion of costates (and hence the related assumptions) and the existence of the continuously differentiable optimal cost function. The idea behind Krotov formulation is that the original optimal control problem is translated into another equivalent optimization problem utilizing the so-called extension principle. The resulting optimization problem is nonconvex, which is generally solved using iterative methods to yield the globally optimal solution. The angle of our attack is to compute a noniterative solution, which is achieved by imposing convexity conditions on the equivalent optimization problem. This convexity imposition lets us compute the optimal control laws without introducing any assumption or using any a priori information and also explains the origin of a Riccati LMI in the literature. To accomplish convexity imposition, the selection of the Krotov function becomes very crucial, which shall be addressed in the future specifically for nonlinear optimal control and distributed optimal control problems.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that they have no conflicts of interest.