Solution to Singular Optimal Control by Backward Differential Flow

Abstract

This paper presents a backward differential flow for solving singular optimal control problems. By using Krotov equivalent transformation, the cost functional is converted to a class of global optimization problems. Some properties of the flow are given to reveal the significant relationship between the dynamic of the flow and the geometry of the feasible set. The proposed method is also used in solving a class of variational problems. Some examples are illustrated.

1. Introduction

In this paper, we are interested in solving the following optimal control problem: (P):min𝐽(𝑢)=^𝑇₀𝑐^𝑇(𝑢̇(0)𝑥+𝑔)𝑑𝑡,s.t.𝑥=𝐴𝑥+Bu,𝑥=𝑥₀,𝑡∈[0,𝑇], (1) where 𝐴∈𝑅^𝑛×𝑛, 𝐵∈𝑅^𝑛×𝑚 are constant matrices, the initial state 𝑥₀ is a constant vector in 𝑅^𝑛, 𝑐∈𝑅^𝑛 is a constant vector, and 𝑔(𝑢) is a polynomial of degree 2𝑙 having the form 𝑔(𝑢)=𝑢₁^2𝑙+𝑢₂^2𝑙+⋯+𝑢_𝑚^2𝑙+𝑓𝑢₁,𝑢₂,…,𝑢_𝑚, (2) where 𝑓(𝑢₁,𝑢₂,…,𝑢_𝑚) is a polynomial of degree less than 2𝑙. We assume that ∇²𝑔(𝑢)≥0. The problem (P) is a singular optimal control problem. The admissible control function 𝑢(⋅), taking values in 𝑅^𝑚, is integrable and bounded on [0,𝑇]. The set of all admissible controls is denoted by Φ. This problem often comes up as a main objective in general optimal control theory [1, 2].

With Krotov [3] equivalent transformation, the optimal control problem (P) can be converted to some auxiliary optimization problems with constraints. A class of backward differential equations [4] relying continuously on time point 𝑡∈[0,𝑇] is introduced for solving these optimization problems.

The rest of the paper is organized as follows. In Sections 2 and 3, with Krotov extension method [3], we convert the optimal control problem (P) to a class of constrained global optimization problems. In Sections 4 and 5, we introduce the theory of backward differential flow [4] for global optimizations. We illustrate the solution to the problem (P) by an example in Section 6. An application of the proposed method in a kind of variational problem is included in Section 7. The last section is the conclusion of the paper.

2. Krotov Equivalent Extension

Let 𝜑(𝑡) be the solution to the following ordinary equation: 𝜑̇(𝑡)+𝐴^𝑇𝜑(𝑡)𝜑(𝑇)=𝑐,=0. (3) The cost functional 𝐽(𝑢) of the problem (P) can be rewritten as follows: 𝐽(𝑢)=^𝑇₀𝑐^𝑇𝑥(𝑡)(𝑢(𝑡=+𝑝))𝑑𝑡_𝑇0𝜑̇^𝑇(𝑡)𝑥(𝑡)+𝜑^𝑇(𝑡)𝐴𝑥(𝑡)+𝜑^𝑇(𝑡)(𝑡)(𝑢(𝑡Bu+𝑔))−𝜑^𝑇(𝑡)(𝑡=Bu)𝑑𝑡_𝑇0𝜑̇^𝑇(𝑡)𝑥(𝑡)+𝜑^𝑇(𝑡)𝑥̇(𝑡)+𝑔(𝑢(𝑡))−𝜑^𝑇(𝑡)Bu(𝑡)𝑑𝑡=𝜑^𝑇(𝑇)𝑥(𝑇)−𝜑^𝑇(0)𝑥₀+^𝑇₀𝑔(𝑢(𝑡))−𝜑^𝑇(𝑡)Bu(𝑡)𝑑𝑡=−𝜑^𝑇(0)𝑥₀+^𝑇₀𝑔(𝑢(𝑡))−𝜑^𝑇(𝑡)Bu(𝑡)𝑑𝑡, (4) noting that 𝜑(𝑇)=0. So we have min_Φ𝐽(𝑢)=−𝜑^𝑇(0)𝑥₀+min_Φ^𝑇₀𝑔(𝑢(𝑡))−𝜑^𝑇(𝑡)Bu(𝑡)𝑑𝑡. (5) It suffices for minimizing 𝐽(𝑢) over Φ to do the following global optimization for each 𝑡∈[0,𝑇]: min_𝑅𝑚𝑔(𝑢)−𝜑^𝑇(𝑡)Bu. (6)

For each 𝑡∈[0,𝑇], let ℎ_𝑡(𝑢)=𝑔(𝑢)−𝜑^𝑇(𝑡)Bu. For given 𝑡, ℎ_𝑡(𝑢) is a polynomial having the same form as 𝑔(𝑢) in (2) except for the linear part. In particular, ∇²ℎ_𝑡(𝑢)=∇²𝑔(𝑢)≥0, (∀𝑡∈[0,𝑇]).

Remark 1. Krotov extension [3] has been used in (5) for finding an optimal control.

3. Global Minimizers of a Polynomial

Let 𝑝(𝑢) be a polynomial over 𝑅^𝑚 having the same form as 𝑔(𝑢) or ℎ_𝑡(𝑢). Here we briefly introduce the main result in [5, 6] for estimating a bound of global minimizers for the polynomial 𝑝(𝑢) over 𝑅^𝑚. For the polynomial 𝑝(𝑢), we have the constrained optimization with the minimum denoted by 𝐼 as follows: 𝐼=min_{1≤𝑗1,𝑗2,…,𝑗2𝑙≤𝑚}𝜕^2𝑙𝑝(0)𝜕𝑢_𝑗1𝜕𝑢_𝑗2⋯𝜕𝑢_𝑗2𝑙𝑠_𝑗1𝑠_𝑗2⋯𝑠_𝑗2𝑙s.t.𝑠²₁+𝑠²₂+⋯+𝑠²_𝑚=1. (7) The following result gives a bound of all global minimizers for the polynomial 𝑝(𝑢) over 𝑅^𝑚.

Theorem A (see [6]). If 𝐼>0, all the global minimizers of the polynomial 𝑝(𝑢) stay inside the ball ‖𝑢‖<𝑎, where ()!𝑎=max1,2𝑙𝑚+𝑚²/2!+⋯+𝑚^2𝑙−1/(2𝑙−1)!𝑀𝐼,𝜕𝑀=max^𝑘𝑝(0)𝜕𝑢_𝑗1⋯𝜕𝑢_𝑗𝑘∶0≤𝑘≤2𝑙−1,1≤𝑗₁,𝑗₂,…,𝑗_𝑘≤𝑚. (8)

If further let 𝑀>max_{𝑡∈[0,𝑇]}‖𝜑^𝑇(𝑡)𝐵‖, then there is a bound 𝑎>0 such that for all 𝑡 the global minimizers of ℎ_𝑡(𝑢) stay inside ‖𝑢‖<𝑎. Therefore, for 𝑡∈[0,𝑇] the optimization problem (6) is equivalent to the following constrained optimization problem: minℎ_𝑡(𝑢)∶‖𝑢‖≤𝑎. (9)

Remark 2. The unconstrained global optimization (6) has been converted to a constrained one here for constructing a backward flow in next section.

4. Canonical Backward Differential Flow

In what follows the polynomial 𝑝(𝑢) still stands for 𝑔(𝑢) or ℎ_𝑡(𝑢) as in the previous section. We keep the assumption ∇²𝑝(𝑢)≥0 as in Section 1. We introduce the theory of canonical backward differential flow [2] for solving the following optimization problem min𝑝(𝑢),s.t.𝑢∈𝐷, (10) where 𝐷={𝑢∈𝑅^𝑚∶‖𝑢‖≤𝑎} with the radius 𝑎 being got in Section 2 such that all minimizers of 𝑝(𝑢) over 𝐷 stay in int𝐷={𝑢∈𝑅^𝑚∶‖𝑢‖<𝑎}.

Since ∇²𝑝(𝑢)≥0(∀𝑢∈𝐷), we have, for 𝜌>0∇²𝑝(𝑢)+𝜌𝐼>0,∀𝑢∈𝐷. (11) For the pair (𝜌,𝑢)∈𝑅⁺×𝐷 with 𝑢≠0 satisfying the following K-T equation: ∇𝑝(𝑢)+𝜌𝑢=0, (12) we define the flow 𝑢(𝜌) near (𝜌,𝑢) by the following ordinary differential equation: 𝑑𝑢𝜌+∇𝑑𝜌²𝑝𝑢𝜌+𝜌𝐼⁻¹𝑢𝜌=0,𝑢𝜌=𝑢. (13) The canonical dual function [4, 7] with respect to the flow 𝑢(𝜌) is defined as follows: 𝑝_𝑑𝜌𝑢𝜌+𝜌=𝑝2𝑢^𝑇𝜌𝑢𝜌−𝑎². (14)

Lemma 3 (see [4]). For a given flow defined by (13), one has 𝑑𝑝_𝑑𝜌=1𝑑𝜌2𝑢^𝑇𝜌𝑢𝜌−𝑎²,𝑑²𝑝_𝑑𝜌𝑑𝜌²𝑑𝑢𝜌=−𝑑𝜌^𝑇∇²𝑝𝑢𝜌𝑑𝑢𝜌+𝜌𝐼.𝑑𝜌 (15)(16)

Lemma 4 (see [4]). Let 𝑢(𝜌) be the flow defined by (13) and 𝑝_𝑑(𝜌) be the corresponding canonical dual function defined by (14), one has(i)for every 𝜌>0, 𝑑²𝑝_𝑑(𝜌)/𝑑𝜌²≤0;(ii)if 𝜌>0, then 𝑑𝑝_𝑑(𝜌)/𝑑𝜌 monotonously decreases in [𝜌,+∞);(iii)if 𝜌>0 and 𝑢(𝜌)∈𝐷, then 𝑝_𝑑(𝜌) monotonously decreases in [𝜌,+∞).

Definition 5 (see [4]). Let 𝑢(𝜌) be the flow defined by (13). When being restricted in (0,𝜌],𝑢(𝜌) is called the backward differential flow.

5. Global Optimization

In this section, we use the backward differential flow to find a global minimizer to the optimization problem (10). In what follows we assume that ∇𝑝(0)≠0.

Since 𝐷 is closed and bounded, there is a 𝜌^∗>0 such that 𝜌^∗>sup_𝐷‖∇𝑝(𝑢)‖ and ∇²𝑝(𝑢)+𝜌^∗𝐼>0, for all 𝑢∈𝐷. By Brown fixed-point theorem [8], we have a nonzero point 𝑢^∗∈int𝐷 such that ∇𝑝𝑢^∗+𝜌^∗𝑢^∗=0,∇²𝑝(𝑢)+𝜌^∗𝐼>0,∀𝑢∈𝐷. (17) Since ∇²𝑝(𝑢)≥0 in 𝑅^𝑚, ∇²𝑝(𝑢)+𝜌𝐼>0 in 𝑅^𝑚 provided that 𝜌>0. It follows from the differential equation, 𝑑𝑢𝜌+∇𝑑𝜌²𝑝𝑢𝜌+𝜌𝐼⁻¹𝑢𝜌=0,𝑢𝜌^∗=𝑢^∗ (18) that a differential flow 𝑢(𝜌) corresponding to the problem (10) exists in (0,+∞). By (14), (15), we see that on the flow for every 𝜌∈(0,+∞)∇𝑝𝑢𝜌+𝜌𝑢𝜌=0,∇²𝑝(𝑢)+𝜌𝐼>0,∀𝑢∈𝑅^𝑚. (19) Since 𝑢^∗∈int𝐷, the flow 𝑢(𝜌)∈int𝐷 as 𝜌≥𝜌^∗ by Lemma 4. We claim that the backward flow 𝑢(𝜌)(0<𝜌≤𝜌^∗) also keeps staying inside 𝐷. If this is not true, then there is a 𝜌∈(0,𝜌^∗) such that ‖𝑢(𝜌)‖=𝑎. Since 𝑝(𝑢)+(𝜌/2)𝑢^𝑇𝑢 is convex, by (19) we deduce that, for 𝑢∈𝐷, 𝑝(𝑢)(𝑢)+𝜌≥𝑝2𝑢^𝑇𝑢−𝑎²≥inf_𝐷𝑝(𝑢)+𝜌2𝑢^𝑇𝑢−𝑎²𝑢𝜌+𝜌=𝑝2𝑢^𝑇𝜌𝑢𝜌−𝑎²=𝑝𝑢𝜌. (20) This contradicts the fact that the minimizer of 𝑝(𝑢) over 𝐷 will stay in int𝐷. By (15), this implies that, for every 𝜌>0, 𝑢=𝑢(𝜌) is the unique point over 𝐷 such that ∇𝑝(𝑢)+𝜌𝑢=0. (21) Consequently, by the classical ordinary differential equation theory, the backward differential flow 𝑢(𝜌) got by (18) can also be got by the following initial value problem: 𝑑𝑢𝜌+∇𝑑𝜌²𝑝𝑢𝜌+𝜌𝐼⁻¹𝑢𝜌=0,𝑢𝜌=𝑢. (22) In conclusion we have reached the following result.

Lemma 6. For every positive parameter 𝜌, there is solely a point 𝑢 located inside 𝐷, such that ∇𝑝(𝑢)+𝜌𝑢=0. (23) Further, the differential flow corresponding to the problem (10) is unique and can be got by the equation 𝑑𝑢𝜌+∇𝑑𝜌²𝑝𝑢𝜌+𝜌𝐼⁻¹𝑢𝜌=0,𝑢𝜌=𝑢. (24)

Remark 7. In other words, the differential flow 𝑢(𝜌) corresponding to the problem (10) does not depend on changing the initial condition.

Theorem 8. Let 𝑢(𝜌) be the backward flow corresponding to the problem (10). Then 𝑢^∗=lim_𝜌→0⁺𝑢(𝜌) exists and 𝑢^∗ is a global minimizer of 𝑝(𝑢) over 𝐷.

Proof. By Lemma 3, the backward flow 𝑢(𝜌) is well defined in (0,+∞) and 𝑢(𝜌)∈𝐷, for all 𝜌>0. Since {𝑢(𝜌)∶𝜌∈(0,+∞)} is bounded, there is a point 𝑢^∗∈𝐷 and a decreasing sequence of positives 𝜌_𝑗(𝑗=1,2,…) such that lim_𝑗→∞𝜌_𝑗=0 and lim_𝑗→∞𝑢(𝜌_𝑗)=𝑢^∗. By Lemma 4, noting that 𝑝_𝑑(𝜌) is increasing monotonously as the positive variable 𝜌 goes to zero monotonously, we have lim_𝜌→0+𝑝_𝑑𝜌=lim_𝑗→∞𝑝_𝑑𝜌_𝑗=lim_𝑗→∞𝑝𝑢𝜌_𝑗+𝜌_𝑗2𝑢𝜌_𝑗^𝑇𝑢𝜌_𝑗−𝑎²=𝑝𝑢^∗. (25) On the other hand, along the flow 𝑢(𝜌), for each 𝜌∈(0,+∞) we have ∇𝑝𝑢𝜌+𝜌2𝑢^𝑇𝜌𝑢𝜌−𝑎²=∇𝑝𝑢𝜌+𝜌𝑢𝜌=0, (26) and for all 𝑢∈𝐷∇²𝑝(𝑢)+𝜌2𝑢^𝑇𝑢−𝑎²=∇²𝑝(𝑢)+𝜌𝐼>0, (27) noting that ∇²𝑝(𝑢)≥0 on 𝐷. Thus, for all 𝑢∈𝐷, 𝑝(𝑢)(𝑢)+𝜌≥𝑝2𝑢^𝑇𝑢−𝑎²≥inf_𝐷𝑝(𝑢)+𝜌2𝑢^𝑇𝑢−𝑎²𝑢𝜌+𝜌=𝑝2𝑢^𝑇𝜌𝑢𝜌−𝑎²=𝑝_𝑑𝜌. (28) Let 𝜌→0⁺, and we have, by (28), (25), for all 𝑢∈𝐷, 𝑝(𝑢)≥lim_𝜌→0+𝑝_𝑑𝜌=𝑝𝑢^∗. (29) We conclude that 𝑢^∗ is a global minimizer of 𝑝(𝑢) over the ball 𝐷.

6. Solution to Singular Optimal Control

To solve the singular optimal control problem (P), by Lemma 6 and Theorem 8, we present the following algorithm.

Algorithm 9. (i) Choose 𝜌>0.
(ii) For 𝑡∈[0,𝑇], solve ∇𝑝(𝑢)−𝐵^𝑇𝜑(𝑡)+𝜌𝑢=0.
(iii) Solve 𝑑𝑢𝜌+∇𝑑𝜌²𝑝𝑢𝜌+𝜌𝐼⁻¹𝑢𝜌=0,𝑢𝜌=𝑢. (30)
(iv) For 𝑡∈[0,𝑇], 𝑢^∗_𝑡=𝑢(0⁺). Go back to (ii).

Since 𝜑(𝑡) is continuous and the solution of the canonical backward differential equation relies on the parameter 𝑡∈[0,𝑇] for the initial value continuously, we see that 𝑢^∗_𝑡 will be dependent of the parameter 𝑡∈[0,𝑇] continuously. It follows from Krotov extension method that the optimal control of (P) is 𝑢(𝑡)∶=𝑢^∗_𝑡,𝑡∈[0,𝑇].

Example 10. Given the polynomial on 𝑅²: 𝑔(𝑢)=𝑢⁴₁+𝑢⁴₂+3𝑢²₁+3𝑢²₂+6𝑢₁𝑢₂+2𝑢₁+2𝑢₂+3, (31) consider the following singular optimal control problem: min𝐽(𝑢)=¹₀𝑐^𝑇(𝑢̇(0)[],𝑥+𝑔)𝑑𝑡,s.t.𝑥=𝐴𝑥+Bu,𝑥=0,𝑡∈0,1 (32) where 𝑐=11^𝑇,.,𝐴=1011𝐵=11−12 (33)
For 𝑡∈[0,1], we have ℎ_𝑡(𝑢)=𝑔(𝑢)+(1−𝑡𝑒^1−𝑡)𝑢₁+(−2+(3−𝑡)𝑒^1−𝑡)𝑢₂. Noting that ∇²𝑔(𝑢)≥0, we have ∇²ℎ_𝑡(𝑢)≥0. By Theorem A, for ℎ_𝑡(𝑢), we have the global optimization problem minℎ_𝑡(𝑢),s.t.‖𝑢‖≤64. (34) By Algorithm 9, for simplicity, choose 𝜌=1 and solve the following algebraic differential equation 𝜌=1,𝑑𝑢₁𝑑𝜌=−12𝑢²₂+6+𝜌𝑢₁−6𝑢₂12𝑢²₁+6+𝜌12𝑢²₂,+6+𝜌−36𝑑𝑢₂𝑑𝜌=−−6𝑢₁+12𝑢²₁+6+𝜌𝑢₂12𝑢²₁+6+𝜌12𝑢²₂,𝑢+6+𝜌−36₁𝜌=𝑢₁,𝑢₂𝜌=𝑢₂,4𝑢³₁+6𝑢₁+6𝑢₂+𝜌𝑢₁=−3+𝑡𝑒^1−𝑡,4𝑢³₂+6𝑢₂+6𝑢₁+𝜌𝑢₂=(𝑡−3)𝑒^1−𝑡, (35) to get the corresponding canonical backward differential flow 𝑢(𝜌)=(𝑢₁(𝜌)𝑢₂(𝜌))^𝑇,𝜌∈(0,1]. For each 𝑡 define 𝑢^∗_𝑡=𝑢(0⁺)=(𝑢₁(0⁺)𝑢₂(0⁺))^𝑇. We get an optimal control for the problem (32): 𝑢(𝑡)=𝑢^∗_𝑡. Table 1 contains some data of approximate value on the optimal control.

Table 1 

Some data on the optimal control for Example 10.

7. Application in a Variational Problem

In this section, we use backward differential flows developed in Sections 4 and 5 to deal with the following variational problem: (P):min_{𝑢(⋅)]∈𝐶[0,1,𝑅1}𝐽(𝑢)=¹₀𝑢⁴(𝑡)+^𝑡₀𝑢(𝑠)𝑑𝑠𝑑𝑡. (36) This kind of problem appears very often in many complex systems from nonconvex analysis of phase transitions to discrete optimization in network design and communication [9, 10].

The functional 𝐽(𝑢) of the variational problem (P) can be rewritten as follows: 𝐽(𝑢)=¹₀𝑢⁴+^𝑡₀𝑢(𝑠)=𝑑𝑠𝑑𝑡₁₀𝑢⁴(𝑡)+(1−𝑡)𝑢(𝑡)𝑑𝑡. (37) For 𝑡∈[0,1], let ℎ_𝑡(𝑢)=𝑢⁴+(1−𝑡)𝑢. Noting that ∇²ℎ_𝑡(𝑢)=12𝑢²≥0, with Algorithm 9, for simplicity, choose 𝜌=1. To get the corresponding canonical backward differential flow 𝑢_𝑡(𝜌),𝜌∈(0,1], we have the following differential equation: 𝑑𝑢=𝑑𝜌−𝑢12𝑢²+𝜌,4𝑢³(1)+𝑢(1)=𝑡−1. (38) We have 𝑢_𝑡0⁺=(𝑡−1)4^1/3. (39) For each 𝑡∈[0,1] defining 𝑢^∗_𝑡=𝑢_𝑡(0⁺), we get the solution to the variational problem (36): 𝑢(𝑡)=𝑢^∗_𝑡=(𝑡−1)4^1/3. (40)

8. Concluding Remarks

In this paper, a new approach to solve singular optimal control is presented. As the first step of this approach, we convert the original optimal control problem to a ball constrained optimization problem. Then a differential equation is established by the K-T equation with the ball constrained nonlinear programming. The main contribution is the development of constructive backward differential flow which can be effectively used for finding a global minimizer. A kind of variational problems can also be solved by this method.