• Views 205
• Citations 0
• ePub 0
• PDF 21
`Mathematical Problems in EngineeringVolume 2019, Article ID 9490512, 17 pageshttps://doi.org/10.1155/2019/9490512`
Research Article

## Linear Quadratic Optimal Control Design: A Novel Approach Based on Krotov Conditions

Indian Institute of Technology Mandi, School of Computing and Electrical Engineering, Himachal Pradesh 175005, India

Received 9 March 2019; Accepted 16 August 2019; Published 13 October 2019

Copyright © 2019 Avinash Kumar and Tushar Jain. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

This paper revisits the problem of synthesizing the optimal control law for linear systems with a quadratic cost. For this problem, traditionally, the state feedback gain matrix of the optimal controller is computed by solving the Riccati equation, which is primarily obtained using calculus of variations- (CoV-) based and Hamilton–Jacobi–Bellman (HJB) equation-based approaches. To obtain the Riccati equation, these approaches require some assumptions in the solution procedure; that is, the former approach requires the notion of costates and then their relationship with states is exploited to obtain the closed-form expression of the optimal control law, while the latter requires a priori knowledge regarding the optimal cost function. In this paper, we propose a novel method for computing linear quadratic optimal control laws by using the global optimal control framework introduced by V. F. Krotov. As shall be illustrated in this article, this framework does not require the notion of costates and any a priori information regarding the optimal cost function. Nevertheless, using this framework, the optimal control problem gets translated to a nonconvex optimization problem. The novelty of the proposed method lies in transforming this nonconvex optimization problem into a convex problem. The convexity imposition results in a linear matrix inequality (LMI), whose analysis is reported in this work. Furthermore, this LMI reduces to the Riccati equation upon imposing optimality requirements. The insights along with the future directions of the work are presented and gathered at appropriate locations in this article. Finally, numerical results are provided to demonstrate the proposed methodology.

#### 1. Introduction

Optimal control is a heavily explored and still developing area of control engineering where the objective is to design a control law so as to optimize (maximize or minimize) a given performance index (cost functional) while driving the states of a dynamical system to zero (regulation problem) or to make output track a reference trajectory (tracking problem) [1]. The generic optimal control problem (GOCP) is given as follows: notation: throughout this article, lowercase alphabets represent scalar quantities, lowercase bold alphabets represent vector quantities, and uppercase alphabets represent matrices, and also, the quantities with “star” in the superscript correspond to the optimal trajectory.

The GOCP computes an optimal control law which minimizes the performance index/cost functionalsubject to the system dynamics to give the desired optimal trajectory . Here, is the running cost, is the terminal cost, is the state vector, and is the control input vector to be designed. Also, and are continuous.

Since, the aforementioned problem corresponds to optimization of the cost functional subject to dynamics of the system considered and possibly constraints on input(s) and/or state(s), the calculus of variations (CoV) is generally employed to address optimal control design problems [1, 2]. The assumption of existence of optimal control is usually the first step while using CoV techniques. Subsequently, the conditions which must be satisfied by such an optimal control law are derived. Hence, only necessary conditions are found, and sufficiency of these conditions is not guaranteed. Furthermore, the obtained control law is usually only locally optimum. Nevertheless, there are results available in the literature which provide restrictions under which the necessary conditions indeed become sufficient and the global optimal control law is obtained [35]. Note that, in solving optimal control design problems, the CoV-based methods use the notion of so-called costates (which are not actually present in the system). Moreover, in the solution procedure, the existence of a linear relationship between the states and the costates is exploited to compute the closed-form expression of the optimal control law (this is particularly true for linear quadratic problems) (see [6] for more details).

Alongside CoV, another tool, namely, dynamic programming (DP) (introduced by Bellman), has also been explored to solve optimal control problems. The application of DP to optimal control design problems for continuous linear systems leads to the celebrated Hamilton–Jacobi–Bellman (HJB) equation which also gives a necessary condition for optimality [1]. Nevertheless, this equation also provides sufficiency under the following mild conditions [2, 7]: there exists a continuously differentiable optimal cost function and the gradient of optimal cost function with respect to the state vector equals the costate which corresponds to the optimal trajectory. For example, consider the optimal control design problem for the system [8] with the performance measure as . For this problem, the optimal cost function is , and hence, the HJB equation is not defined at because of nondifferentiability of . From the aforementioned observations, a solution method for optimal control design which does not require these conditions is desirable. As shall be demonstrated in this article, the Krotov solution methodology is indeed such a methodology. In fact, this methodology provides sufficient conditions for existence of the global optimal control law without using the notion of costates and any a priori information regarding the optimal cost function [2].

The rest of this article is organized as follows: In Section 2, the preliminaries of linear quadratic optimal control problems are discussed and the solution methodologies based on the calculus of variations- (CoV-) based method and Hamilton–Jacobi–Bellman (HJB) equation-based method are outlined. The assumptions as encountered in these approaches are also discussed. In Section 3, the background literature of Krotov sufficient conditions and their application to the problems considered are detailed. This section also discusses a number of insights which result while solving the considered optimal control problems. These insights are gathered as remarks at appropriate locations in this section. In Section 4, the analysis of LMI obtained upon convexity imposition is presented. The Krotov iterative method is also discussed in brief in this section. In Section 5, the proposed method is demonstrated through numerical examples. Finally, the concluding remarks and future scope of the work are presented in Section 6.

#### 2. Preliminaries and Problem Formulation

In this section, solution procedures of linear quadratic regulation (LQR) and linear quadratic tracking (LQT) problems using CoV- and HJB equation-based approaches are briefly discussed in order to highlight the assumptions used in these approaches.

##### 2.1. OCP 1 (Finite Horizon LQR Problem)

Compute an optimal control law which minimizes the quadratic performance index/cost functionalsubject to the system dynamics and drives the states of the system to zero (regulation). Here, is given, is free, and is fixed. Also,

The solution using the CoV technique comprises the following four major steps:(i)Formulation of Hamiltonian function: the Hamiltonian for the considered problem is given aswhere is the costate vector.(ii)Obtaining the optimal control law using the first-order necessary condition: the optimal control law is obtained as(iii)Use of state and costate dynamics and a transformation to connect states and costates for all : the boundary conditions (i.e., being fixed and being free) lead to the following boundary condition on : . Then, the transformationto connect costate and state vectors is used to compute the optimal control law as(iv)Obtaining the matrix differential Riccati equation: finally, taking the derivative of equation (5) and substituting the state and costate relationship, the following matrix differential Riccati equation (MDRE) is obtained which must satisfy for all :

Furthermore, the solution using the HJB equation requireswhere is the optimal cost function and is the optimal control law. To solve (8), the boundary condition is used with assumed to bewhere is a real, symmetric, positive-definite matrix to be determined. Substituting (9) into (8), we get

This equation is valid for any if

Finally, , and thus, the solution is the same as that obtained using CoV. Summarizing the above, the global optimal control law is given bywhere is the solution ofwith the boundary condition .

##### 2.2. OCP 2 (Finite Horizon LQT Problem)

Compute an optimal control law which minimizes the following quadratic performance index/cost functional:where subject to the system dynamicssuch that the output tracks the desired reference trajectory . Here, is the error vector, is given, is free, and is fixed. Also,

Similarly to the solution of LQR, the CoV- and HJB equation-based approaches yield the optimal control law aswhere and satisfywith boundary conditions and , respectively. Note that, in the HJB approach, the optimal cost function has to be guessed.

Although the CoV- and HJB equation-based approaches as described above are widely employed for solving OCPs, there are some assumptions associated with these approaches in their solution procedure. Specifically, the CoV-based approach uses the notion of costates and their relationship with states for all time (5) to compute the optimal control law. Similarly, the HJB equation-based approach requires the existence of the continuously differentiable optimal cost function, and its gradient with respect to the state is the costate corresponding to the optimal trajectory [7]. Thus, the information about the optimal cost function must be known a priori. The angle of our attack is to synthesize an optimal control law using Krotov sufficient conditions, where the above issues are not encountered in the solution procedure. However, it is well known that the control law using these conditions is synthesized through an iterative procedure. The main nontrivial issue to be tackled is how to obtain noniterative solutions of optimal control problems using Krotov conditions. The next section answers this question for linear quadratic optimal control problems.

#### 3. Computation of Optimal Control Laws

In this section, solutions of the LQR and LQT problems using Krotov sufficient conditions are detailed.

##### 3.1. Krotov Sufficient Conditions in Optimal Control

The underlying idea behind Krotov sufficient conditions for global optimality of control processes is the total decomposition of the original OCP with respect to time using the so-called extension principle [2].

###### 3.1.1. Extension Principle

The essence of the extension principle is to replace the original optimization problem with complex relations and/or constraints with a simpler one such that they are excluded in the new problem definition but the solution of the new problem still satisfies the discarded relations [11].

Consider a scalar-valued functional defined over a set (i.e., ) and the optimization problem as follows.

Problem 1. Find such that where . Instead of solving Problem 1, another equivalent optimization problem is solved. Let L denote the equivalent representation of the original cost functional. Then, a new problem is formulated over , a superset of , as follows.Equivalent Problem 1. Find such that where .The equivalent problem is also called the extension of the original problem. The method of choosing of the equivalent functional L is not unique, and the selection is generally made according to specifications of the problem under consideration. This freedom in the selection of the equivalent functional can be exploited to tackle the generic nonconvex optimization problems. Also, it is necessary to ensure that , so that the optimizer is actually the optimizer of original Problem 1 [2]. The idea behind the extension principle is illustrated in Figure 1. Clearly, the application of the extension principle requires appropriate selection of the equivalent functional L and the set .