Solution to an Optimal Control Problem via Canonical Dual Method

Abstract

The analytic solution to an optimal control problem is investigated using the canonical dual method. By means of the Pontryagin principle and a transformation of the cost functional, the optimal control of a nonconvex problem is obtained. It turns out that the optimal control can be expressed by the costate via canonical dual variables. Some examples are illustrated.

1. Introduction

Consider the following optimal control problem (primal problem (𝒫) in short):

s.t.𝑥̇=(𝒫𝐴)(𝑡)m𝑥in+𝐵^𝑇0([𝑡𝐹)𝑢(,𝑥)𝑥+(𝑃0)(=𝑢)]𝑥𝑑₀𝑡,(1)𝑡∈[0,𝑇],‖𝑢‖≤∈1,𝑅^𝑛,(2) where 𝐹(⋅) is continuous on 𝑅^𝑛, and 𝑃(⋅) is twice continuously differentiable in 𝑅^𝑚. An admissible control, taking values on the unit ball 𝐷∶={𝑢∈𝑅^𝑚∣‖𝑢‖≤1}, is integrable or piecewise continuous on [0,𝑇]. In (2) we assume that 𝐴(𝑡),𝐵(𝑡) are continuous matrix functions in 𝐶([0,𝑇],𝑅^𝑛×𝑛) and 𝐶([0,𝑇],𝑅^𝑛×𝑚), respectively. This problem often comes up as a main objective in general optimal control theory [1].

By the classical control theory [2], we have the following Hamilton-Jacobi-Bellman function:

𝐻(𝑡,𝑥,𝑢,𝜆)=𝜆^∗(𝐴(𝑡)𝑥+𝐵(𝑡)𝑢)+𝐹(𝑥)+𝑃(𝑢).(3) The state and costate systems are

𝜆̇𝑥=̇=−𝐻𝐻_𝜆𝑥(𝑡,𝑥,𝑢,𝜆)=𝐴(𝑡)𝑥+𝐵(𝑡)𝑢,𝑥(0)=𝑥₀,(𝑡,𝑥,𝑢,𝜆)=−𝐴^∗𝜆−∇𝐹(𝑥),𝜆(𝑇)=0.(4)

In general, it is difficult to obtain an analytic form of the optimal feedback control for the problem (1)-(2). It is well known that, in the case of unconstraint, if 𝑃(𝑢) is a positive definite quadratic form and 𝐹(𝑥) is a positive semidefinite quadratic form, then a perfect optimal feedback control is obtained by the solution of a Riccati matrix differential equation. The primal goal of this paper is to present an analytic solution to the optimal control problem (𝒫).

We know from the Pontryagin principle [1] that if the control 𝑢̂ is an optimal solution to the problem (𝒫), with 𝑥̂(⋅) and 𝜆̂(⋅) denoting the state and costate corresponding to 𝑢(̂⋅), respectively, then 𝑢̂ is an extremal control, that is, we have

𝜆̇̂𝑥=̇̂=−𝐻𝐻_𝜆𝑥𝑡,𝑥̂,𝑢,̂𝜆̂=𝐴(𝑡)𝑥̂+𝐵(𝑡)𝑢,̂𝑥̂(0)=𝑥₀,(5)𝐻𝑡,𝑥̂(𝑡),𝑢̂(𝑡𝑡,)𝑥,̂,𝜆𝑢̂(,̂𝑡𝜆)̂==−m𝐴in^∗_{‖𝑢‖≤1}𝜆̂−∇𝐹(𝑥̂),𝜆̂(𝑇)=0,(6)𝐻a.e𝑡.,𝑡𝑥̂∈(𝑡)[,0𝑢,,𝑇𝜆]̂(.𝑡),(7)

By means of the Pontryagin principle and the dynamic programming theory, many numerical algorithms have been suggested to approximate the solution to the problem (𝒫) (see, [3–5]). This is due to the nonlinear integrand in the cost functional. It is even difficult for the case of 𝑃(𝑢) being nonconvex on the unit ball 𝐷 in 𝑅^𝑚. We know that when 𝑃(𝑢) is nonconvex on the unit ball 𝐷, sometimes the optimal control of the problem (𝒫) may exist. Let us see the following simple example for 𝑛=𝑚=1:

(𝒫)min¹⁰𝑥−1s.t.𝑥̇=𝑥+𝑢,𝑥(0)=0,𝑡2∈𝑢²[0𝑑,𝑇𝑡,],|𝑢|≤1.(8) In fact, it is easy to see that 𝑢(̂𝑡)≡−1;𝑡∈[0,𝑇] is an optimal control.

In this paper, we consider 𝑃(𝑢) to be nonconvex. If the optimal control of the problem (𝒫) exists, we solve the problem (1) to find the optimal control which is an expression of the costate. We see that, with respect to 𝑢, the minimization in (7) is equivalent to the following global nonconvex optimization over a sphere:

min_{‖𝑢‖≤1}𝑃(𝑢)+𝜆̂(𝑡)^∗𝐵(𝑡)𝑢,a.e.𝑡∈[0,𝑇],(9) when 𝑃(𝑢) is a nonconvex quadratic function, the problem (9) can be solved completely by the canonical dual transformation [6–8]. In [9], the global concave optimization over a sphere is solved by use of a differential system with the canonical dual function. Because the Pontryagin principle is a necessary condition for a control to be optimal, it is not sufficient for obtaining an optimal control to solve only the optimization (9). In this paper, combing the method given in [6, 9] with the Pontryagin principle, we solve problem (1)-(2) which has nonconvex integrand on the control variable in the cost functional and present the optimal control expressed by the costate via canonical dual variables.

2. Global Optimization over a Sphere

In this section we present a differential flow to deal with the global optimization, which is used to find the optimal control expressed by the costate in the next section. Here we use the method in our another paper (see [9]).

In what follows we consider the function 𝑃(𝑥) to be twice continuously differentiable and nonconvex on the unit ball in 𝑅^𝑚. Define the set

𝐺=𝜌>0∣∇²𝑃(𝑥)+𝜌𝐼>0,𝑥^∗𝑥≤1.(10) Since 𝑃(𝑥) is nonconvex and ∇²𝑃(𝑥) is bounded on 𝐷∶={𝑥∈𝑅^𝑚𝑥^∗𝑥≤1}, 𝐺 is an open interval (𝜌,+∞) for the nonnegative real number 𝜌 depending on 𝑃(𝑥). Let 𝜌^∗∈𝐺 and 𝑥̃∈{𝑥^∗𝑥≤1} satisfy the following KKT equation:

∇𝑃(𝑥̃)+𝜌^∗𝑥̃=0.(11) We focus on the flow 𝑥̂(𝜌) defined near 𝜌^∗ by the following backward differential equation:

𝑑𝑥̂𝑑𝜌+∇²𝑃(𝑥̂)+𝜌𝐼𝑥̂𝜌⁻¹_∗𝑥̂=0,𝜌∈𝜌^∗−𝛿,𝜌^∗,(12)=𝑥̃.(13) The flow 𝑥̂(𝜌) can be extended to wherever 𝜌∈𝐺∩(0,𝜌^∗] [10]. The dual function [6] with respect to a given flow 𝑥̂(𝜌) is defined as

𝑃_𝑑(𝜌)=𝑃(𝑥̂(𝜌))+𝜌2𝑥̂^∗(𝜌)𝑥̂(𝜌)−𝜌2.(14) We have

𝑑𝑥𝑑̂𝜌(𝜌)^∗∇²𝑃(𝑥̂(𝜌))+𝜌𝐼𝑑𝑥̂(𝜌)=−1𝑑𝜌2𝑑𝑥̂^∗(𝜌)𝑥̂(𝜌)𝑑𝜌=−𝑑²𝑃_𝑑(𝜌̂)𝑑𝜌²≥0.(15) Consequently

𝑑²𝑃_𝑑(𝜌)𝑑𝜌²≤0.(16)

It means that 𝑑𝑃_𝑑(𝜌)/𝑑𝜌 decreases when 𝜌 increases in 𝐺. If, for a 𝜌̂∈𝐺, 𝑑𝑃_𝑑(𝜌̂)/𝑑𝜌≤0, then 𝑑𝑃_𝑑(𝜌)/𝑑𝜌≤0 for 𝜌∈𝐺∩[𝜌̂,∞). Therefore,

𝑃_𝑑(𝜌̂)≥𝑃_𝑑(𝜌),(17) as long as 𝜌≥𝜌̂.

Theorem 1. If the flow 𝑥̂(𝜌),𝜌∈𝐺∩(0,𝜌^∗], defined by (11)–(13), passes through a boundary point of the ball 𝐷={𝑥∈𝑅^𝑚‖𝑥‖≤1} at 𝜌̂∈𝐺, that is, [𝑥̂(𝜌̂)]^∗𝑥̂(𝜌̂)=1,𝜌̂∈𝐺∩0,𝜌^∗,(18) then 𝑥̂ is a global minimizer of 𝑃(𝑥) over the ball 𝐷. Further one has min_𝐷𝑃(𝑥)=𝑃(𝑥̂)=𝑃_𝑑(𝜌̂)=max_𝜌≥𝜌̂𝑃^𝑑(𝜌).(19)

Proof. Since 𝜌̂∈𝐺, 𝜌̂>0. For each 𝑥∈𝐷 and whenever 𝜌≥𝜌̂ we have 𝑃(𝑥)≥𝑃(𝑥)+𝜌2𝑥^∗≥inf_𝐷𝑥−1𝑃(𝑥)+𝜌2𝑥^∗𝑥−1=𝑃_𝑑(𝜌).(20) By (17), (18), we have 𝑃(𝑥)≥max_𝜌≥𝜌̂𝑃^𝑑=𝑃(𝑥̂(𝜌̂))+(𝜌𝜌)=𝑃_𝑑(𝜌̂)2(𝑥̂(𝜌̂))^∗𝑥̂(𝜌̂)−1=𝑃(𝑥̂(𝜌̂)).(21) Thus min_𝐷𝑃(𝑥)=max_𝜌≥𝜌̂𝑃^𝑑(𝜌).(22) This concludes the proof of Theorem 1.

To illustrate the canonical dual method, let us present several examples as follows.

Example 2. Let us consider the following one-dimensional concave minimization problem: 𝑝^∗=min𝑃(𝑥)=−112𝑥⁴s.t.𝑥₂−𝑥²+𝑥,≤1.(23) We have 𝑃^(𝑥)=−13𝑥³−2𝑥+1,𝑃^(𝑥)=−𝑥²−2<0,∀𝑥²≤1.(24) By choosing 𝜌^∗=10, we solve the following equation in {𝑥²<1}: −13𝑥³−2𝑥+1+10𝑥=0(25) to get a solution 𝑥̃=−0.1251. Next we solve the following boundary value problem of the ordinary differential equation: 𝑑𝑥(𝜌)𝑑𝜌=𝑥²(𝜌𝑥)+(𝜌)2−𝜌,𝑥𝜌^∗=−0.1251,𝜌≤10.(26) To find a parameter such that 𝑥²(𝜌)=1,(27) we get 𝜌̂=103,(28) which satisfies 𝑃^(𝑥)+𝜌̂=𝑃^(𝑥)+103>0,∀𝑥²≤1.(29) Let 𝑥(10/3) be denoted by 𝑥̂. To find the value of 𝑥̂, we compute the solution of the following algebra equation: −13𝑥³−2𝑥+1+103𝑥=0,𝑥²=1(30) and get 𝑥̂=−1. It follows from Theorem 1 that 𝑥̂=−1 is the global minimizer of 𝑃(𝑥) over [−1,1].

Example 3. We now consider the nonconvex minimization problem: 𝑝^∗=min𝑃(𝑥)=13𝑥³s.t.𝑥₂+2𝑥,≤1.(31) By choosing 𝜌^∗=√72, we solve the following equation in {𝑥²<1}: 𝑥²+2+√72𝑥=0(32) to get a solution 𝑥̃=−2/(4+3√2). Next we solve the following boundary value problem of the ordinary differential equation: 𝑥̇=−𝑥2𝑥+𝑡,𝑡≤√𝑥√72,72=−24+3√2.(33) To find a parameter such that 𝑥²(𝜌)=1,(34) we get 𝜌̂=3,(35) which satisfies 𝑃^(𝑥)+𝜌̂=𝑃^(𝑥)+3=2𝑥+3>0,∀𝑥²≤1.(36) Let 𝑥(3) be denoted by 𝑥̂. To find the value of 𝑥̂, we compute the solution of the following algebra equation: 𝑥²+2+3𝑥=0,𝑥²=1(37) and get 𝑥̂=−1. It follows from Theorem 1 that 𝑥̂=−1 is the global minimizer of 𝑃(𝑥) over [−1,1].

Example 4. Given a symmetric matrix 𝐺∈𝑅^𝑚×𝑚 and a nonzero vector 𝑓∈𝑅^𝑚, let 𝑃(𝑥)=(1/2)𝑥^∗𝐺𝑥−𝑓^∗𝑥 be a nonconvex quadratic function. Consider the following global optimization problem over a sphere: min𝑃(𝑥)∶=12𝑥^∗s.t.𝑥_∗𝐺𝑥−𝑓^∗𝑥,𝑥≤1.(38) Suppose that 𝐺 has 𝑝≤𝑚 distinct eigenvalues 𝑎₁<𝑎₂<⋯<𝑎_𝑝. Since 𝑃(𝑥)=(1/2)𝑥^∗𝐺𝑥−𝑓^∗𝑥 is nonconvex, 𝑎₁<0. Let us choose a large 𝜌^∗>−𝑎₁ such that 0<‖‖𝐺+𝜌^∗𝐼⁻¹𝑓‖‖<1.(39) By solving the boundary value problem of ordinary differential equation 𝑑𝑥𝑑𝜌=−(𝐺+𝜌𝐼)⁻¹𝑥,𝜌^∗=𝐺+𝜌^∗𝐼⁻¹𝑓,𝜌≤𝜌^∗,(40) we get the unique solution 𝑥(𝜌)=(𝐺+𝜌𝐼)⁻¹𝑓,𝜌≤𝜌^∗.(41) Since 𝐺 is symmetric, there exists an orthogonal matrix 𝑅 such that 𝑅𝐺𝑅^∗=𝐷∶=(𝑎_𝑖𝛿_𝑖𝑗) (a diagonal matrix) and correspondingly 𝑅𝑓=𝑔∶=(𝑔_𝑖) (a vector). By (41), we have 𝑥^∗(𝜌)𝑥(𝜌)=𝑓^∗(𝐺+𝜌𝐼)⁻²𝑓=^𝑝_𝑖=1𝑔^2𝑖𝑎_𝑖+𝜌².(42) Since 𝑓^∗(𝐺+𝜌^∗𝐼)⁻²𝑓<1 and lim_{𝜌>−𝑎1,𝜌→−𝑎1}^𝑝_𝑖=1𝑔^2𝑖𝑎_𝑖+𝜌²=+∞,(43) there exists 𝜌̂∈(−𝑎₁,𝜌^∗) uniquely such that 𝑥^∗(𝜌̂)𝑥(𝜌̂)=𝑓^∗(𝐺+𝜌̂𝐼)⁻²𝑓=^𝑝_𝑖=1𝑔^2𝑖𝑎_𝑖+𝜌̂²=1.(44) By Theorem 1, we see that 𝑥(𝜌̂)=(𝐺+𝜌̂𝐼)⁻¹𝑓 is a global minimizer of the problem.

3. Find an Analytic Solution to the OptimalControl Problem

In this section, we consider 𝐴(𝑡),𝐵(𝑡) in problem (1)-(2) to be constant matrices, 𝐹(𝑥)=𝑐^∗𝑥 and

𝑃(𝑢)=12𝑢^∗𝐺𝑢−𝑏^∗𝑢,(45)

where 𝑐∈𝑅^𝑛×1,𝑏∈𝑅^𝑚×1, and 𝐺(∈𝑅^𝑚×𝑚) is a symmetric matrix. Suppose that 𝐺 has 𝑝≤𝑚 distinct eigenvalues 𝑎₁<𝑎₂<⋯<𝑎_𝑝 and 𝑎₁<0. Moreover, we need the following basic assumption:

rank𝐵^∗,𝑏>rank𝐵^∗.(∗)

We consider the following optimal control problem:

(𝒫)min𝐽(𝑢)=^𝑇0𝑐^∗𝑥+12𝑢^∗s.t.𝑥̇=𝐴𝑥+𝐵𝑢,𝑥(0)=𝑥₀𝐺𝑢−𝑏^∗𝑢𝑑𝑡,(46),𝑡∈[0,𝑇],‖𝑢‖≤1.(47)

To solve the above problem, we define the function 𝜙(𝑡,𝑥)=𝜓^∗(𝑡)𝑥, where 𝜓(𝑡) is the solution to the following Cauchy boundary value problem of the ordinary differential equation:

𝜓̇(𝑡)=𝜓(−𝑇𝐴)^∗=𝜓0(𝑡.)+𝑐,((4489)) By comparing (48)-(49) with (6) in terms of this special problem (46)-(47), we see that

𝜓(𝑡)=−𝜆(𝑡),a.e.𝑡∈[0,𝑇].(50)

Noting that 𝜓(𝑇)=0 and 𝑥(0)=𝑥₀, we have

𝐽(𝑢)=^𝑇0𝑐^∗𝑥+12𝑢^∗=𝐺𝑢−𝑏^∗𝑢𝑑𝑡_𝑇0=𝜓̇(𝑡)+𝐴^∗𝜓(𝑡)^∗𝑥+21𝑢^∗𝐺𝑢−𝑏^∗𝑢𝑑𝑡_𝑇0=𝜓̇^∗(𝑡)𝑥+𝜓(𝑡)^∗𝐴𝑥+12𝑢^∗𝐺𝑢−𝑏^∗𝑢𝑑𝑡_𝑇0−𝜓𝜓̇^∗((𝑡𝑡))_∗𝑥+𝜓(𝑡)^∗(𝐴𝑥+𝐵𝑢)𝐵𝑢+12𝑢^∗=𝐺𝑢−𝑏^∗𝑢𝑑𝑡_𝑇0−𝜓𝜓̇^∗((𝑡𝑡))_∗𝑥(𝑡)+𝜓(𝑡)^∗𝑥̇(𝑡)𝐵𝑢+12𝑢^∗=𝐺𝑢−𝑏^∗𝑢𝑑𝑡_𝑇0=+𝜙(𝑇,𝜙𝑥̇((𝑡𝑇,𝑥))(𝑡−))𝜙−(𝜓0,(𝑥𝑡)(^∗0𝐵)𝑢)_𝑇0+21𝑢^∗𝐺𝑢−𝑏^∗𝑢𝑑𝑡=−𝜙(0,12𝑥𝑢(^∗0𝐺))𝑢+−𝑏^∗_𝑇0𝑢−𝜓(𝑡)^∗𝐵𝑢𝑑𝑡12𝑢^∗𝐺𝑢−𝑏^∗𝑢−𝜓(𝑡)^∗𝐵𝑢𝑑𝑡.(51) Thus,

min𝐽(𝑢)=+−𝜙mi(n0,𝑥_𝑇0(0))12𝑢^∗𝐺𝑢−𝑏^∗𝑢−𝜓(𝑡)^∗𝐵𝑢𝑑𝑡.(52)

Consequently, we deduce that, for almost every 𝑡 in [0,𝑇], the optimal control is

𝑢̂(𝑡)=argmin_{𝑢∗𝑢≤1}12𝑢^∗𝐺𝑢−𝑏^∗𝑢−𝜓(𝑡)^∗𝐵𝑢.(53) By the relation between 𝜓(𝑡) and the costate in (50), for given 𝑡∈[0,𝑇], we need to solve the following nonconvex optimization:

min12𝑢^∗𝐺s𝑢.t.−𝑢_∗𝑏−𝐵^∗𝜆(𝑡)^∗𝑢,𝑢≤1.(54) It follows from the basic assumption (∗) that 𝑏−𝐵^𝑇𝜆(𝑡)≠0 for each 𝑡∈[0,𝑇]. By Example 4 and (53), for almost every 𝑡 in [0,𝑇], we have

𝑢̂(𝑡)=𝐺+𝜌_𝑡𝐼⁻¹𝑏−𝐵^∗𝜆(𝑡),(55) with the dual variable 𝜌_𝑡>−𝑎₁ satisfying

𝑏−𝐵^∗𝜆(𝑡)^∗𝐺+𝜌_𝑡𝐼⁻²𝑏−𝐵^∗𝜆(𝑡)=1.(56) We define the function 𝜌(𝜆) with respect to 𝜆 by the following equation:

𝑏−𝐵^∗𝜆^∗(𝐺+𝜌𝜌(𝜆(𝜆))>𝐼)−𝑎⁻²₁𝑏−𝐵^∗𝜆=1,(57) and obtain an analytic solution to the optimal control problem via a costate expression

𝑢̂=(𝐺+𝜌(𝜆)𝐼)⁻¹𝑏−𝐵^∗𝜆.(58)

On the other hand, by the solution of the Cauchy boundary value problem of the ordinary differential equation (48)-(49), we have

𝜆(𝑡)==−𝑒𝜓_𝐴∗𝑇(𝑡)^0𝑇−𝑡=𝑒_{𝐴∗(𝑇−𝑡)}𝑒^{−𝐴∗𝑡}𝑒^{−𝐴∗𝑠}𝑑𝑠𝑐^0𝑇−𝑡𝑒^{−𝐴∗𝑠}𝑑𝑠𝑐.(59)

Example 5. Consider the following optimal control problem: (𝒫)min¹⁰𝑥−1𝑥(0)=0s,.t.𝑡𝑥̇∈=[𝑥0,+1]𝑢2,,𝑢|²𝑢|≤𝑑𝑡1.(60) This is a simple case of (46),(47). We have 𝐺=−1,𝑐=1,𝑏=0,𝐴=1,𝐵=1,𝑇=1. By (59), we have 𝜆(𝑡)=𝑒^1−𝑡^01−𝑡𝑒^−𝑠𝑑𝑠=𝑒^1−𝑡−1≠0(𝑡≠1).(61) To find an analytic solution of the optimal control problem, we solve the equation (𝜌−1)⁻²𝜆²(𝑡)=1,𝜌>1(62) to get 𝜌=1+||𝜆(𝑡)||=1+𝑒^1−𝑡−1.(63) By (58), we obtain an analytic solution of the optimal control problem which can be expressed as 𝑢̂(𝑡)==(𝜌𝑒−1)⁻¹[−𝜆(𝑡)_1−𝑡]−1⁻¹1−𝑒^1−𝑡=−1,(𝑡≠1).(64)

4. Concluding Remarks

In this paper, a new approach to optimal control problems has been investigated using the canonical dual method. Some nonlinear and nonconvex problems can be solved by global optimizations, and therefore, the differential flow defined by the KKT equation (see (11)) can produce an analytic solution of the optimal control problem. Meanwhile, by means of the canonical dual function, an optimality condition is proved (see Theorem 1). The global optimization problem is solved by a backward differential equation with an equality condition (see (12), (18)). More research needs to be done for the development of applicable canonical dual theory.

Acknowledgment

This research was partly supported by the National Science Foundation of China under grants No. 10671145.