Indefinite LQ Control for Discrete-Time Stochastic Systems via Semidefinite Programming

Abstract

This paper is concerned with a discrete-time indefinite stochastic LQ problem in an infinite-time horizon. A generalized stochastic algebraic Riccati equation (GSARE) that involves the Moore-Penrose inverse of a matrix and a positive semidefinite constraint is introduced. We mainly use a semidefinite-programming- (SDP-) based approach to study corresponding problems. Several relations among SDP complementary duality, the GSARE, and the optimality of LQ problem are established.

1. Introduction

Stochastic linear quadratic (LQ) control problem was pioneered by Wonham [1] and has become one of the most popular research field of modern control theory; see, for example, [2–12] and the references therein. In the most early literature about stochastic LQ issue, it is always assumed that the control weighting matrix 𝑅 is positive definite and the state weighting matrix 𝑄 is positive semidefinite as the deterministic LQ problem does. However, a surprising fact was found that, different from deterministic LQ problem, for a stochastic LQ modeled by a stochastic Itô-type differential system, the original LQ optimization may still be well posed even if the cost weighting matrices 𝑄 and 𝑅 are indefinite [5]. Follow-up research was carried out, and a lot of important results were obtained. In [6–9], continuous-time indefinite stochastic LQ control problem was studied. For the discrete-time case, there have been some works. For example, the system with only control-dependent noises was studied in [10]. The finite time and infinite horizon indefinite stochastic LQ control problem with state- and control-dependent noises were, respectively, studied in [11, 12].

In this paper, we study discrete-time indefinite stochastic LQ control problem over an infinite time horizon. The system involves multiplicative noises in both the state and the control. We mainly use the SDP approach introduced in [9, 13] to discuss the corresponding problem. We first introduce a generalized stochastic algebraic Riccati equation (GSARE) that involves the Moore-Penrose inverse of a matrix. The potential relations among LQ problem, SDP, and GSARE are studied. What we have obtained extends the results of [9] from continuous-time case to discrete-time case.

The remainder of this paper is organized as follows. In Section 2, we formulate the discrete-time indefinite stochastic LQ problem and present some preliminaries including generalized stochastic algebraic Riccati equation, SDP, and some lemmas. Section 3 contains the main results. Some relations among the optimality of the LQ problem, the complementary optimal solutions of the SDP and its dual problem, and the solvability of the GSARE are established. Some comments are given in Section 4.

Notations. ℛ^𝑛: 𝑛-dimensional Euclidean space. ℛ^𝑛×𝑚: the set of all 𝑛×𝑚 matrices. 𝒮^𝑛: the set of all 𝑛×𝑛 symmetric matrices. 𝐴^′: the transpose of matrix 𝐴. 𝐴≥0(𝐴>0): 𝐴 is positive semidefinite (positive definite). 𝐼: the identity matrix. ℛ: the set of all real numbers. 𝑁∶={0,1,2,…} and 𝑁_𝑡∶={0,1,2,…,𝑡}. Tr(𝑀): the trace of a square matrix 𝑀. 𝒜^adj: the adjoint mapping of a mapping 𝒜.

2. Preliminaries

2.1. Problem Statement

Consider the following discrete-time stochastic system:𝑥()(𝑡)(𝑡)+[(𝑡)(𝑡𝑤(𝑡),𝑥(0)𝑡+1=𝐴𝑥+𝐵𝑢𝐶𝑥+𝐷𝑢)]=𝑥₀,𝑡=0,1,2,…, (2.1) where 𝑥(𝑡)∈ℛ^𝑛,  𝑢(𝑡)∈ℛ^𝑚 are, respectively, the system state and control input. 𝑥₀∈ℛ^𝑛 is the initial state and 𝑤(𝑡)∈ℛ is the noise. 𝐴,𝐶∈ℛ^𝑛×𝑛 and 𝐵,𝐷∈ℛ^𝑛×𝑚 are constant matrices. {𝑤(𝑡),𝑡∈𝑁} is a sequence of real random variables defined on a complete probability space (Ω,ℱ,ℱ_𝑡,𝑃) with ℱ_𝑡=𝜎{𝑤(𝑠)∶𝑠∈𝑁_𝑡}, which is a wide sense stationary, second-order process with 𝐸[𝑤(𝑡)]=0 and 𝐸[𝑤(𝑠)𝑤(𝑡)]=𝛿_𝑠𝑡, where 𝛿_𝑠𝑡 is the Kronecker function. 𝑢(𝑡) belongs to ℒ²_ℱ(ℛ^𝑚), the space of all ℛ^𝑚-valued, ℱ_𝑡-adapted measurable processes satisfying 𝐸(∑^∞_𝑡=0‖𝑢(𝑡)‖²)<∞. We assume that the initial state 𝑥₀ is independent of the noise 𝑤(𝑡),𝑡∈𝑁.

We first give the following definitions.

Definition 2.1. System (2.1) is called mean square stabilizable if there exists a feedback control 𝑢(𝑡)=𝐾𝑥(𝑡) such that for any initial state 𝑥₀, the closed-loop system 𝑥()=()𝑥(𝑡)+()𝑥(𝑡)𝑤(𝑡),𝑥(0)𝑡+1𝐴+𝐵𝐾𝐶+𝐷𝐾=𝑥₀,𝑡=0,1,2,…, (2.2) is asymptotically mean square stable, that is, the corresponding state of (2.2) satisfies lim_𝑡→∞𝐸‖𝑥(𝑡)‖²=0, where 𝐾∈ℛ^𝑚×𝑛 is a constant matrix.

For system (2.1), we define the admissible control set 𝑈_𝑎𝑑=𝑢(𝑡)∈ℒ²_ℱℛ^𝑚,𝑢(𝑡)ismeansquarestabilizingcontrol. (2.3) The cost functional associated with system (2.1) is 𝐽𝑥₀,𝑢=^∞_𝑡=0𝐸𝑥^′(𝑡)𝑄𝑥(𝑡)+𝑢^′(𝑡)𝑅𝑢(𝑡), (2.4) where 𝑄 and 𝑅 are symmetric matrices with appropriate dimensions and may be indefinite. The LQ optimal control problem is to minimize the cost functional 𝐽(𝑥₀,𝑢) over 𝑢∈𝑈_𝑎𝑑. We define the optimal value function as 𝑉𝑥₀=inf_{𝑢∈𝑈𝑎𝑑}𝐽𝑥₀,𝑢. (2.5) Since the weighting matrices 𝑄 and 𝑅 may be indefinite, the LQ problem is called an indefinite stochastic LQ control problem.

Definition 2.2. The LQ problem is called well posed if −∞<𝑉𝑥₀<∞,∀𝑥₀∈ℛ^𝑛. (2.6) If there exists an admissible control 𝑢^∗ such that 𝑉(𝑥₀)=𝐽(𝑥₀,𝑢^∗), the LQ problem is called attainable and 𝑉(𝑥₀) is the optimal cost value. 𝑢^∗(𝑡),𝑡∈𝑁, is called an optimal control, and 𝑥^∗(𝑡),𝑡∈𝑁, corresponding to 𝑢^∗(𝑡) is called the optimal trajectory.

Stochastic algebraic Riccati equation (SARE) is a primary tool in solving stochastic LQ control problems. In [12], the following discrete SARE:−𝑃+𝐴^′𝑃𝐴+𝐶^′𝑃𝐶+𝑄−𝐴^′𝑃𝐵+𝐶^′𝑃𝐷𝑅+𝐵^′𝑃𝐵+𝐷^′𝑃𝐷⁻¹𝐵^′𝑃𝐴+𝐷^′𝑃𝐶=0,𝑅+𝐵^′𝑃𝐵+𝐷^′𝑃𝐷>0, (2.7) was studied. The constraint that 𝑅+𝐵^′𝑃𝐵+𝐷^′𝑃𝐷>0 is demanded in (2.7). In fact, the corresponding LQ problem may have optimal control even if the condition is not satisfied. In this paper, we introduce the following generalized stochastic algebraic Riccati equation (GSARE), ℛ(𝑃)≡−𝑃+𝐴^′𝑃𝐴+𝐶^′𝑃𝐶+𝑄−𝐴^′𝑃𝐵+𝐶^′𝑃𝐷𝑅+𝐵^′𝑃𝐵+𝐷^′𝑃𝐷⁺𝐵^′𝑃𝐴+𝐷^′𝑃𝐶=0,𝑅+𝐵^′𝑃𝐵+𝐷^′𝑃𝐷≥0, (2.8) which weakens the positive definiteness constraint of 𝑅+𝐵^′𝑃𝐵+𝐷^′𝑃𝐷 to positive semidefiniteness constraint and replaces the inverse by Moore-Penrose inverse. Hence, (2.8) is an extension of (2.7).

2.2. Semidefinite Programming

In this subsection, we will introduce SDP and its dual. SDP is a special conic optimization problem and is defined as follows.

Definition 2.3 (see [14]). Suppose that 𝒱 is a finite-dimensional vector space with an inner product ⟨⋅,⋅⟩_𝒱 and 𝒮 is a space of block diagonal symmetric matrices with an inner product ⟨⋅,⋅⟩_𝒮. 𝒜∶𝒱→𝒮 is a linear mapping, and 𝐴₀∈𝒮. The following optimization problem: min⟨𝑐,𝑥⟩_𝒱,s.t.𝐴(𝑥)=𝒜(𝑥)+𝐴₀≥0, (2.9) is called a semidefinite programming (SDP). From convex duality, the dual problem associated with the SDP is defined as max−𝐴₀,𝑍_𝒮,s.t.𝒜^adj=𝑐,𝑍≥0. (2.10) In the context of duality, we refer to the SDP (2.9) as the primal problem associated with (2.10).

Consider the following SDP problem: (𝑃)(𝑃),(𝑃)=maxTrs.t.𝐴−𝑃+𝐴^′𝑃𝐴+𝐶^′𝑃𝐶+𝑄𝐴^′𝑃𝐵+𝐶^′𝐵𝑃𝐷_′𝑃𝐴+𝐷^′𝑃𝐶𝑅+𝐵^′𝑃𝐵+𝐷^′𝑃𝐷≥0. (2.11) By the definition of SDP, we can get the dual problem of (2.11).

Proposition 2.4. The dual problem of (2.11) can be formulated as (𝐷)minTr(𝑄𝑆+𝑅𝑇),s.t.−𝑆+𝐴𝑆𝐴^′+𝐶𝑆𝐶^′+𝐵𝑈𝐴^′+𝐷𝑈𝐶^′+𝐴𝑈^′𝐵^′+𝐶𝑈^′𝐷^′+𝐵𝑇𝐵^′+𝐷𝑇𝐷^′+𝐼=0,𝑍=𝑆𝑈^′𝑈𝑇≥0. (2.12)

Proof. The objective of the primal problem can be rewritten as maximizing ⟨𝐼,𝑃⟩_𝒮𝑛. The dual variable 𝑍=^𝑆𝑈′_𝑈𝑇≥0, where (𝑆,𝑇,𝑈)∈𝒮^𝑛×𝒮^𝑚×ℛ^𝑚×𝑛. The LMI constraint in the primal problem can be represented as 𝐴(𝑃)=𝒜(𝑃)+𝐴₀=−𝑃+𝐴^′𝑃𝐴+𝐶^′𝑃𝐶𝐴^′𝑃𝐵+𝐶^′𝐵𝑃𝐷_′𝑃𝐴+𝐷^′𝑃𝐶𝐵^′𝑃𝐵+𝐷^′+𝑃𝐷𝑄00𝑅. (2.13) According to the definition of adjoint mapping, we have ⟨𝒜(𝑃),𝑍⟩_{𝒮𝑛+𝑚}=⟨𝑃,𝒜^adj(𝑍)⟩_𝒮𝑛, that is, Tr[𝒜(𝑃)𝑍]=Tr[𝑃𝒜^adj(𝑍)]. It follows 𝒜^adj(𝑍)=−𝑆+𝐴𝑆𝐴^′+𝐶𝑆𝐶^′+𝐵𝑈𝐴^′+𝐷𝑈𝐶^′+𝐴𝑈^′𝐵^′+𝐶𝑈^′𝐷^′+𝐵𝑇𝐵^′+𝐷𝑇𝐷^′. By Definition 2.3, the objective of the dual problem is to minimize ⟨𝐴₀,𝑍⟩_{𝒮𝑛+𝑚}=Tr(𝐴₀𝑍)=Tr(𝑄𝑆+𝑅𝑇). On the other hand, we will state that the constraints of the dual problem (2.10) are equivalent to the constraints of (2.12). Obviously, 𝒜^adj(𝑍)=−𝐼 is equivalent to the equality constraint of (2.12). This ends the proof.

The primal problem (2.9) is said to satisfy the Slater condition if there exists a primal feasible solution 𝑥⁰ such that 𝐴(𝑥⁰)>0, that is, the primal problem (2.9) is strictly feasible. The dual problem (2.10) is said to satisfy the Slater condition if there is a dual feasible solution 𝑍⁰ satisfying 𝑍⁰>0, that is, the dual problem (2.10) is strictly feasible.

Let 𝑝^∗ and 𝑑^∗ denote the optimal values of SDP (2.9) and the dual SDP (2.10), respectively. Let 𝐗_opt and 𝐙_opt denote the primal and dual optimal sets. Then, we have the following proposition (see [13, Theorem  3.1]).

Proposition 2.5. 𝑝^∗=𝑑^∗ if either of the following conditions holds.(1)The primal problem (2.9) satisfies Slater condition.(2)The dual problem (2.10) satisfies Slater condition.

If both conditions hold, the optimal sets 𝐗_opt and 𝐙_opt are nonempty. In this case, a feasible point 𝑥 is optimal if and only if there is a feasible point 𝑍 satisfying the complementary slackness condition: 𝐴(𝑥)𝑍=0. (2.14)

2.3. Some Definitions and Lemmas

The following definitions and lemmas will be used frequently in this paper.

Definition 2.6. For any matrix 𝑀, there exists a unique matrix 𝑀⁺, called the Moore-Penrose inverse of 𝑀, satisfying 𝑀𝑀⁺𝑀=𝑀,𝑀⁺𝑀𝑀⁺=𝑀⁺,𝑀𝑀^+′=𝑀𝑀⁺,𝑀⁺𝑀^′=𝑀⁺𝑀. (2.15)

Lemma 2.7 (extended Schur’s lemma). Let matrices 𝑀=𝑀^′, 𝑁, and 𝑅=𝑅^′ be given with appropriate dimensions. Then, the following conditions are equivalent:(1)𝑀−𝑁𝑅⁺𝑁^′≥0, 𝑅≥0, and 𝑁(𝐼−𝑅𝑅⁺)=0,(2)_𝑁^𝑀𝑁_′𝑅≥0, (3)^𝑅𝑁′_𝑁𝑀≥0.

Lemma 2.8 (see [7]). For a symmetric matrix 𝑆, we have(1)𝑆⁺=(𝑆⁺)^′, (2)𝑆≥0 if and only if 𝑆⁺≥0,(3)𝑆𝑆⁺=𝑆⁺𝑆.

Lemma 2.9 (see [12]). In system (2.1), suppose 𝑇∈𝑁 is given, and 𝑃(𝑡)∈𝒮^𝑛,𝑡=0,1,…,𝑇+1, is an arbitrary family of matrices, then, for any 𝑥(0)∈ℛ^𝑛, we have ^𝑇_𝑡=0𝐸𝑥(𝑡)𝑢(𝑡)^′𝑄[𝑃(𝑡𝑥(𝑡)𝑢(𝑡))]=𝐸𝑥^′(𝑇+1)𝑃(𝑇+1)𝑥(𝑇+1)−𝑥^′(0)𝑃(0)𝑥(0), (2.16) where 𝑄[𝑃(𝑡=(𝑡))]−𝑃+𝐴^′𝑃(𝑡+1)𝐴+𝐶^′𝑃(𝑡+1)𝐶𝐴^′𝑃(𝑡+1)𝐵+𝐶^′𝑃()𝐷𝐵𝑡+1_′𝑃(𝑡+1)𝐴+𝐷^′𝑃(𝑡+1)𝐶𝐵^′𝑃(𝑡+1)𝐵+𝐷^′𝑃(𝑡+1)𝐷. (2.17)

Lemma 2.10. System (2.1) is mean square stabilizable if and only if one of the following conditions holds.(1)There are a matrix 𝐾 and a symmetric matrix 𝑃>0 such that −𝑃+(𝐴+𝐵𝐾)𝑃(𝐴+𝐵𝐾)^′+(𝐶+𝐷𝐾)𝑃(𝐶+𝐷𝐾)^′<0. (2.18) Moreover, the stabilizing feedback control is given by 𝑢(𝑡)=𝐾𝑥(𝑡).(2)For any matrix 𝑌>0, there is a matrix 𝐾 such that the following matrix equation: −𝑃+(𝐴+𝐵𝐾)𝑃(𝐴+𝐵𝐾)^′+(𝐶+𝐷𝐾)𝑃(𝐶+𝐷𝐾)^′+𝑌=0 (2.19) has a unique positive definite solution 𝑃>0. Moreover, the stabilizing feedback control is given by 𝑢(𝑡)=𝐾𝑥(𝑡).(3)The dual problem (𝐷) satisfies the Slater condition.

Proof. (1) and (2) can be derived from Proposition  2.2 in [15]. (3) is a discrete edition of Theorem  6 in [7]. The proof is similar to Theorem  6 in [7] and is omitted.

To this end, we need the following assumptions throughout the paper.

Assumption 2.11. System (2.1) is mean square stabilizable.

Assumption 2.12. The feasible set of (𝑃) is nonempty.

3. Main Results

In this section, we will establish the relationship among the optimality of the LQ problem, the SDP, and the GSARE.

The following theorem reveals the relation between the SDP complementary optimal solutions and the GSARE.

Theorem 3.1. If a feasible solution of (𝑃), 𝑃^∗, satisfies ℛ(𝑃^∗)=0, and the feedback control 𝑢(𝑡)=𝐾^∗𝑥(𝑡)=−𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷⁺𝐵^′𝑃^∗𝐴+𝐷^′𝑃^∗𝐶𝑥(𝑡),𝑡∈𝑁, (3.1) is stabilizing, then there exist complementary optimal solutions of (𝑃) and (𝐷). In particular, 𝑃^∗ is optimal to (𝑃), and there is a complementary dual optimal solution 𝑍^∗ of (𝐷), such that 𝑆^∗>0.

Proof. By the stability assumption of the control 𝑢(𝑡)=𝐾^∗𝑥(𝑡) and Lemma 2.10, the equation −𝑌+𝐴+𝐵𝐾^∗𝑌𝐴+𝐵𝐾^∗′+𝐶+𝐷𝐾^∗𝑌𝐶+𝐷𝐾^∗′+𝐼=0 (3.2) has a positive solution 𝑌^∗>0. Let 𝑆^∗=𝑌^∗,𝑈^∗=𝐾^∗𝑆^∗=𝐾^∗𝑌^∗,𝑇^∗=𝐾^∗𝑈^∗′=𝐾^∗𝑌^∗𝐾^∗′, (3.3) that is, 𝑍^∗=𝑆^∗𝑈^∗′𝑈^∗𝑇^∗=𝑌^∗𝑌^∗𝐾^∗′𝐾^∗𝑌^∗𝐾^∗𝑌^∗𝐾^∗′=𝐾𝐼0_∗𝐼𝑌^∗000𝐼𝐾^∗′0𝐼≥0. (3.4) By (3.2) and (3.3), we have −𝑆^∗+𝐴𝑆^∗𝐴^′+𝐶𝑆^∗𝐶^′+𝐵𝑈^∗𝐴^′+𝐷𝑈^∗𝐶^′+𝐴𝑈^∗′𝐵^′+𝐶𝑈^∗′𝐷^′+𝐵𝑇^∗𝐵^′+𝐷𝑇^∗𝐷^′+𝐼=0, (3.5) which shows 𝑍^∗ is a feasible solution of (𝐷). 𝐴(𝑃^∗)≥0 because 𝑃^∗ is a feasible solution of (𝑃). By Lemmas 2.7 and 2.8, 𝐴^′𝑃^∗𝐵+𝐶^′𝑃^∗𝐷=𝐴^′𝑃^∗𝐵+𝐶^′𝑃^∗𝐷𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷⁺=𝐴^′𝑃^∗𝐵+𝐶^′𝑃^∗𝐷𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷⁺𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷. (3.6) In addition, we have −𝑃^∗+𝐴^′𝑃^∗𝐴+𝐶^′𝑃^∗𝐶+𝑄=𝐾^∗′𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝐾^∗ (3.7) by ℛ(𝑃^∗)=0 and 𝐾^∗=−(𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷)⁺(𝐵^′𝑃^∗𝐴+𝐷^′𝑃^∗𝐶). Therefore, we have 𝐴𝑃^∗𝑍^∗=−𝑃^∗+𝐴^′𝑃^∗𝐴+𝐶^′𝑃^∗𝐶+𝑄𝐴^′𝑃^∗𝐵+𝐶^′𝑃^∗𝐷𝐵^′𝑃^∗𝐴+𝐷^′𝑃^∗𝐶𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝑆^∗𝑈^∗′𝑈^∗𝑇^∗=𝐼−𝐾^∗′ℛ𝑃0𝐼^∗00𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝐼0−𝐾^∗𝐼𝑆^∗𝑈^∗′𝑈^∗𝑇^∗=𝐼−𝐾^∗′=.0𝐼00000000 (3.8) Obviously, 𝑃^∗ and 𝑍^∗ are complementary optimal solutions to (𝑃) and (𝐷).𝑃^∗ is optimal to (𝑃), and 𝑍^∗ is optimal to (𝐷).𝑆^∗>0 is trivial because 𝑆^∗=𝑌^∗>0.

In above, the assumption that the control in (3.1) is stabilizing is not automatically satisfied. The following theorem reveals that we can obtain a stabilizing feedback control by the dual SDP.

Theorem 3.2. Suppose that 𝑍=^𝑆𝑈′_𝑈𝑇 is a feasible solution of (𝐷) with 𝑆>0, then the feedback control 𝑢(𝑡)=𝑈𝑆⁻¹𝑥(𝑡) is stabilizing.

Proof. First, we have 𝑍≥0 because 𝑍 is feasible to (𝐷). By Lemma 2.7, the inequality 𝑇−𝑈𝑆⁻¹𝑈^′≥0 holds. By simple calculations, we have 𝐴𝑆𝐴^′+𝐵𝑈𝐴^′+𝐴𝑈^′𝐵^′+𝐵𝑈𝑆⁻¹𝑈^′𝐵^′=𝐴+𝐵𝑈𝑆⁻¹𝑆𝐴+𝐵𝑈𝑆^−1′,𝐶𝑆𝐶^′+𝐷𝑈𝐶^′+𝐶𝑈^′𝐷^′+𝐷𝑈𝑆⁻¹𝑈^′𝐷^′=𝐶+𝐷𝑈𝑆⁻¹𝑆𝐶+𝐷𝑈𝑆^−1′. (3.9) Hence, 0=−𝑆+𝐴𝑆𝐴^′+𝐶𝑆𝐶^′+𝐵𝑈𝐴^′+𝐷𝑈𝐶^′+𝐴𝑈^′𝐵^′+𝐶𝑈^′𝐷^′+𝐵𝑇𝐵^′+𝐷𝑇𝐷^′+𝐼≥−𝑆+𝐴𝑆𝐴^′+𝐶𝑆𝐶^′+𝐵𝑈𝐴^′+𝐷𝑈𝐶^′+𝐴𝑈^′𝐵^′+𝐶𝑈^′𝐷^′+𝐵𝑈𝑆⁻¹𝑈^′𝐵^′+𝐷𝑈𝑆⁻¹𝑈^′𝐷^′+𝐼>−𝑆+𝐴+𝐵𝑈𝑆⁻¹𝑆𝐴+𝐵𝑈𝑆^−1′+𝐶+𝐷𝑈𝑆⁻¹𝑆𝐶+𝐷𝑈𝑆^−1′. (3.10) Above inequality shows (2.18) has a positive definite solution 𝑆>0 with 𝐾=𝑈𝑆⁻¹. According to Lemma 2.10, 𝑢(𝑡)=𝐾𝑥(𝑡)=𝑈𝑆⁻¹𝑥(𝑡) is stabilizing.

The following theorem shows the relationship between the optimality of the LQ problem and the solution of GSARE.

Theorem 3.3. If LQ problem (2.1)–(2.5) is attainable with respect to any 𝑥₀∈ℛ^𝑛, then (𝑃) must have an optimal solution 𝑃^∗ such that ℛ(𝑃^∗)=0.

Proof. Since the LQ problem is attainable, then the optimal value must be of the quadratic form [16]: inf_{𝑢∈𝑈𝑎𝑑}𝐽𝑥₀,𝑢=𝑥^′₀𝑀𝑥₀,∀𝑥₀∈ℛ^𝑛. (3.11) Let (𝑥^∗(⋅),𝑢^∗(⋅)) be an optimal pair for the initial state 𝑥₀. Let 𝑇→∞ and 𝑃(𝑡)=𝑃 in (2.16), where 𝑃 is an any feasible solution of (𝑃), then we have 𝑥^′₀𝑃𝑥₀+^∞_𝑡=0𝐸𝑥(𝑡)𝑢(𝑡)^′𝑄(𝑃)𝑥(𝑡)𝑢(𝑡)=0. (3.12) Then, a completion square means 𝐽𝑥₀,𝑢^∗=^∞_𝑡=0𝐸𝑥^∗′(𝑡)𝑄𝑥^∗(𝑡)+𝑢^∗′(𝑡)𝑅𝑢^∗(𝑡)=𝑥^′₀𝑃𝑥₀+^∞_𝑡=0𝐸𝑢^∗(𝑡)−𝐾𝑥^∗(𝑡)^′𝑅+𝐵^′𝑃𝐵+𝐷^′𝑃𝐷𝑢^∗(𝑡)−𝐾𝑥^∗(𝑡)+𝑥^∗′(𝑡)ℛ(𝑃)𝑥^∗(𝑡), (3.13) where 𝐾=−(𝑅+𝐵^′𝑃𝐵+𝐷^′𝑃𝐷)⁺(𝐵^′𝑃𝐴+𝐷^′𝑃𝐶). Since 𝑃 is feasible to (𝑃), we have 𝑅+𝐵^′𝑃𝐵+𝐷^′𝑃𝐷≥0 and ℛ(𝑃)≥0 by Lemma 2.7. Then, the inequality 𝑥^′₀𝑀𝑥₀≡𝐽𝑥₀,𝑢^∗≥𝑥^′₀𝑃𝑥₀ (3.14) holds for any feasible solution 𝑃 to (𝑃). This shows that 𝑀 must be optimal to (𝑃). Moreover, taking 𝑃=𝑀 in (3.13) and considering 𝐽(𝑥₀,𝑢^∗)=𝑥^′₀𝑀𝑥₀, we know that 𝐸𝑥^∗′(𝑡)ℛ(𝑀)𝑥^∗(𝑡)=0 for 𝑡∈𝑁. Setting 𝑡=0 and noticing that 𝑥₀ is arbitrary, it follows that ℛ(𝑀)=0.
Below, we will show 𝑀 is a feasible solution of (𝑃). We consider the following SDP and its dual under a perturbation 𝜀>0:𝑃_𝜀maxTr(𝑃),s.t.−𝑃+𝐴^′𝑃𝐴+𝐶^′𝑃𝐶+𝑄+𝜀𝐼𝐴^′𝑃𝐵+𝐶^′𝐵𝑃𝐷_′𝑃𝐴+𝐷^′𝑃𝐶𝑅+𝜀𝐼+𝐵^′𝑃𝐵+𝐷^′𝐷𝑃𝐷≥0,_𝜀minTr[(𝑄+𝜀𝐼)𝑆+(𝑅+𝜀𝐼)𝑇],s.t.−𝑆+𝐴𝑆𝐴^′+𝐶𝑆𝐶^′+𝐵𝑈𝐴^′+𝐷𝑈𝐶^′+𝐴𝑈^′𝐵^′+𝐶𝑈^′𝐷^′+𝐵𝑇𝐵^′+𝐷𝑇𝐷^′+𝐼=0,𝑍=𝑆𝑈^′𝑈𝑇≥0. (3.15)(3.16) Obviously, (𝑃_𝜀) satisfies the Slater condition because we assume that the feasible set of (𝑃) is nonempty and (𝐷_𝜀) also satisfies the Slater condition by the mean square stabilizability assumption and Lemma 2.10. Hence, the complementary optimal solutions exist by Proposition 2.5. Take any dual feasible solution 𝑍⁰=^{𝑆0𝑈0′}_𝑈0𝑇0. By the weak duality in conic optimization problems, we have Tr(𝑃)≤Tr(𝑄+𝜀𝐼)𝑆⁰+(𝑅+𝜀𝐼)𝑇⁰. (3.17) Let 𝑃⁰ be a feasible solution of (𝑃), then 𝑃⁰ is feasible to (𝑃_𝜀) for all 𝜀≥0. Similar to Theorem  10 in [7], we conclude that, for any 𝜀>0, there exists the unique optimal solution of (𝑃_𝜀), denoted by 𝑃^∗_𝜀, and 𝑃^∗_𝜀≥𝑃⁰.
Together with (3.17), we know that 𝑃^∗_𝜀 are contained in a compact set with 0≤𝜀≤𝜀₀ (𝜀₀>0 is a constant). Then, take a convergent subsequence satisfying lim_𝑖→∞𝑃^∗_𝜀𝑖=𝑃^∗₀ with 𝜀_𝑖→0 as 𝑖→∞. Obviously, 𝑃^∗₀ is feasible to (𝑃) because the feasible region of (𝑃_𝜀) monotonically shrinks as 𝜀↓0. Define the perturbed cost functional 𝐽_𝜀𝑥₀,𝑢=^∞_𝑡=0𝐸𝑥^′(𝑡)𝑄_𝜀𝑥(𝑡)+𝑢^′(𝑡)𝑅_𝜀𝑢(𝑡), (3.18) where 𝑅_𝜀=𝑅+𝜀𝐼,𝑄_𝜀=𝑄+𝜀𝐼. By (3.13), we have 𝐽_𝜀𝑥₀,𝑢=𝑥^′₀𝑃^∗_𝜀𝑥₀+^∞_𝑡=0𝐸[𝑢(𝑡)−𝐾_𝜀𝑥(𝑡)]^′𝑅_𝜀+𝐷^′𝑃^∗_𝜀𝐷+𝐵^′𝑃^∗_𝜀𝐵𝑢(𝑡)−𝐾_𝜀𝑥(𝑡)+𝑥^′(𝑡)ℛ_𝜀𝑃^∗_𝜀𝑥(𝑡), (3.19) for any 𝑢∈𝑈_𝑎𝑑, where 𝐾_𝜀=−(𝑅_𝜀+𝐵^′𝑃^∗_𝜀𝐵+𝐷^′𝑃^∗_𝜀𝐷)⁺(𝐵^′𝑃^∗_𝜀𝐴+𝐷^′𝑃^∗_𝜀𝐶) and ℛ_𝜀(𝑃^∗_𝜀) is the form of ℛ(𝑃^∗_𝜀) with 𝑄 and 𝑅 replaced by 𝑄_𝜀 and 𝑅_𝜀. Then, by Theorems  10 and 12 in [7],inf_{𝑢∈𝑈𝑎𝑑}𝐽_𝜀𝑥₀,𝑢=𝑥^′₀𝑃^∗_𝜀𝑥₀. (3.20) Furthermore, 𝑥^′₀𝑃^∗_𝜀𝑖𝑥₀=inf_{𝑢∈𝑈𝑎𝑑}𝐽_𝜀𝑖𝑥₀,𝑢≥inf_{𝑢∈𝑈𝑎𝑑}𝐽𝑥₀,𝑢=𝑥^′₀𝑀𝑥₀. (3.21) Taking limit, we have 𝑥^′₀𝑃^∗₀𝑥₀≥𝑥^′₀𝑀𝑥₀. On the other hand, 𝑥^′₀𝑀𝑥₀≥𝑥^′₀𝑃^∗₀𝑥₀ because 𝑃^∗₀ is feasible to (𝑃) and (3.14). So 𝑀=𝑃^∗₀. The feasibility of 𝑀 is proved. The proof is completed.

The following theorem studies the converse of Theorem 3.3.

Theorem 3.4. If a feasible solution of (𝑃), 𝑃^∗, satisfies ℛ(𝑃^∗)=0 and the feedback control 𝑢^∗(𝑡)=−(𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷)⁺(𝐵^′𝑃^∗𝐴+𝐷^′𝑃^∗𝐶)𝑥(𝑡) is stabilizing, then it must be optimal for LQ problem (2.1)–(2.5).

Proof. For any 𝑢∈𝑈_𝑎𝑑, we have 𝐽𝑥₀,𝑢=𝑥^′₀𝑃^∗𝑥₀+^∞_𝑡=0𝐸𝑢(𝑡)−𝐾^∗𝑥(𝑡)^′𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝑢(𝑡)−𝐾^∗𝑥(𝑡) (3.22) by (3.13) and ℛ(𝑃^∗)=0, where 𝐾^∗=−(𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷)⁺(𝐵^′𝑃^∗𝐴+𝐷^′𝑃^∗𝐶). Because 𝑢^∗(𝑡)=𝐾^∗𝑥(𝑡) is stabilizing, 𝑢^∗(𝑡) must be optimal.

The following theorem shows we can get the optimal feedback control by SDP dual optimal solution.

Theorem 3.5. Assume that (P) and (D) have complementary optimal solutions 𝑃^∗ and 𝑍^∗ with 𝑆^∗>0. Then, ℛ(𝑃^∗)=0 and LQ problem (2.1)–(2.5) has an attainable optimal feedback control given by 𝑢^∗(𝑡)=𝑈^∗(𝑆^∗)⁻¹𝑥^∗(𝑡).

Proof. From the proof of Theorem 3.1, we have 𝐴𝑃^∗=𝐼−𝐾^∗′ℛ𝑃0𝐼^∗00𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝐼0−𝐾^∗𝐼, (3.23) where 𝐾^∗=−(𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷)⁺(𝐵^′𝑃^∗𝐴+𝐷^′𝑃^∗𝐶). By complementary slackness condition 𝐴(𝑃^∗)𝑍^∗=0 and the invertibility of 𝐼−𝐾^∗′0𝐼, we have ℛ𝑃^∗00𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝐼0−𝐾^∗𝐼𝑆^∗𝑈^∗′𝑈^∗𝑇^∗=ℛ𝑃^∗𝑆^∗ℛ𝑃^∗𝑈^∗′−𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝐾^∗𝑆^∗−𝑈^∗−𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝐾^∗𝑈^∗′−𝑇^∗=.0000 (3.24) So ℛ(𝑃^∗)𝑆^∗=0,ℛ(𝑃^∗)𝑈^∗′=0. On the other hand, 𝑇^∗−𝑈^∗(𝑆^∗)⁺𝑈^∗′≥0,𝑆^∗≥0 and 𝑈^∗=𝑈^∗𝑆^∗(𝑆^∗)⁺ from 𝑍^∗≥0 and Lemma 2.7. From the equality constraint in (2.12) and the above results, we have 0=ℛ𝑃^∗−𝑆^∗+𝐴𝑆^∗𝐴^′+𝐶𝑆^∗𝐶^′+𝐵𝑈^∗𝐴^′+𝐷𝑈^∗𝐶^′+𝐴𝑈^∗′𝐵^′+𝐶𝑈^∗′𝐷^′+𝐵𝑇^∗𝐵^′+𝐷𝑇^∗𝐷^′+𝐼ℛ𝑃^∗≥ℛ𝑃^∗𝐴𝑆^∗𝐴^′+𝐶𝑆^∗𝐶^′+𝐵𝑈^∗𝐴^′+𝐷𝑈^∗𝐶^′+𝐴𝑈^∗′𝐵^′+𝐶𝑈^∗′𝐷^′+𝐵𝑈^∗𝑆^∗+𝑈^∗′𝐵^′+𝐷𝑈^∗𝑆^∗+𝑈^∗′𝐷^′+𝐼ℛ𝑃^∗=ℛ𝑃^∗2+ℛ𝑃^∗𝐶𝑆^∗+𝐷𝑈^∗𝑆^∗+𝐶𝑆^∗+𝐷𝑈^∗′+𝐴𝑆^∗+𝐵𝑈^∗𝑆^∗+𝐴𝑆^∗+𝐵𝑈^∗′ℛ𝑃^∗≥ℛ𝑃^∗2. (3.25) The last inequality holds because (𝑆^∗)⁺≥0 from Lemma 2.8. It follows that ℛ(𝑃^∗)=0.
For any 𝑢∈𝑈_𝑎𝑑, by (3.13), we get𝐽𝑥₀,𝑢=𝑥^′₀𝑃𝑥₀+^∞_𝑡=0𝐸[𝑢(𝑡)−𝐾𝑥(𝑡)]^′𝑅+𝐵^′𝑃𝐵+𝐷^′𝑃𝐷[𝑢(𝑡)−𝐾𝑥(𝑡)]+𝑥^′(𝑡)ℛ(𝑃)𝑥(𝑡), (3.26) where 𝑃 is any feasible solution of (𝑃) and 𝐾=−(𝑅+𝐵^′𝑃𝐵+𝐷^′𝑃𝐷)⁺(𝐵^′𝑃𝐴+𝐷^′𝑃𝐶).ℛ(𝑃)≥0 because of the feasibility of 𝑃. Then, 𝐽𝑥₀,𝑢≥𝑥^′₀𝑃𝑥₀. (3.27) On the other hand, 𝑢^∗(𝑡)=𝑈^∗(𝑆^∗)⁻¹𝑥^∗(𝑡) is stabilizing by Theorem 3.2. Let 𝑢(𝑡)=𝑢^∗(𝑡) and 𝑃=𝑃^∗ in (3.26), then it follows that 𝐽𝑥₀,𝑢^∗=𝑥^′₀𝑃^∗𝑥₀+^∞_𝑡=0𝐸𝑢^∗(𝑡)−𝐾^∗𝑥^∗(𝑡)^′𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝑢^∗(𝑡)−𝐾^∗𝑥^∗(𝑡), (3.28) where 𝐾^∗=−(𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷)⁺(𝐵^′𝑃^∗𝐴+𝐷^′𝑃^∗𝐶). Below we prove 𝐽(𝑥₀,𝑢^∗)=𝑥^′₀𝑃^∗𝑥₀. Applying complementary slackness condition 𝐴(𝑃^∗)𝑍^∗=0 and above proof, we have ℛ𝑃^∗00𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝐼0−𝐾^∗𝐼𝑆^∗𝑈^∗′𝑈^∗𝑇^∗=ℛ𝑃^∗𝑆^∗𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝑈^∗+𝐵^′𝑃^∗𝐴+𝐷^′𝑃^∗𝐶𝑆^∗ℛ𝑃^∗𝑈^∗′𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝑇+𝐵^′𝑃^∗𝐴+𝐷^′𝑃^∗𝐶𝑈^∗′=.0000 (3.29) Hence, (𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷)𝑈^∗=−(𝐵^′𝑃^∗𝐴+𝐷^′𝑃^∗𝐶)𝑆^∗. Then, 𝑢^∗(𝑡)−𝐾^∗𝑥^∗(𝑡)^′𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝑢^∗(𝑡)−𝐾^∗𝑥^∗(𝑡)=𝑢^∗′(𝑡)𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝑢^∗(𝑡)+2𝑢^∗′(𝑡)𝐵^′𝑃^∗𝐴+𝐷^′𝑃^∗𝐶𝑥^∗(𝑡)+𝑥^∗′(𝑡)𝐴^′𝑃^∗𝐵+𝐶^′𝑃^∗𝐷𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷⁺𝐵^′𝑃^∗𝐴+𝐷^′𝑃^∗𝐶𝑥^∗(𝑡)=𝑢^∗′(𝑡)𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝑢^∗(𝑡)−2𝑢^∗′(𝑡)𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝑈^∗𝑆^∗−1𝑥^∗(𝑡)+𝑥^∗′(𝑡)𝑆^∗−1𝑈^∗′𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝑈^∗𝑆^∗−1𝑥^∗(𝑡)=𝑢^∗(𝑡)−𝑈^∗𝑆^∗−1𝑥^∗(𝑡)^′𝑅+𝐵^′𝑃^∗𝐵+𝐷^′𝑃^∗𝐷𝑢^∗(𝑡)−𝑈^∗𝑆^∗−1𝑥^∗(𝑡)=0. (3.30) It follows from (3.27) and (3.28) that 𝐽𝑥₀,𝑢^∗=𝑥^′₀𝑃^∗𝑥₀≤𝐽𝑥₀,𝑢,∀𝑢∈𝑈_𝑎𝑑. (3.31) The optimality of 𝑢^∗(𝑡) is proved.

4. Conclusion

In this paper, we use the SDP approach to study discrete-time indefinite stochastic LQ control problem. Some relations are given and are summarized as follows. The condition that LQ problem is attainable can induce that (𝑃) has an optimal solution 𝑃^∗ satisfying GSARE (Theorem 3.3). Theorems 3.4 and 3.5 give two suffcient conditions for LQ problem attainability by GSARE and complementary optimal solutions of (𝑃) and (𝐷). Moreover, by dual SDP, we can get stabilized feedback control (Theorem 3.2). What we have obtained can be viewed as a discrete-time version of [9]. Of course, there are many open problems to be solved. For instance, the indefinite LQ problems for Markovian jumps or time-variant system merit further study.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (61174078), Specialized Research Fund for the Doctoral Program of Higher Education (20103718110006), and Key Project of Natural Science Foundation of Shandong Province (ZR2009GZ001).