Discrete-Time Indefinite Stochastic LQ Control via SDP and LMI Methods

Abstract

This paper studies a discrete-time stochastic LQ problem over an infinite time horizon with state-and control-dependent noises, whereas the weighting matrices in the cost function are allowed to be indefinite. We mainly use semidefinite programming (SDP) and its duality to treat corresponding problems. Several relations among stability, SDP complementary duality, the existence of the solution to stochastic algebraic Riccati equation (SARE), and the optimality of LQ problem are established. We can test mean square stabilizability and solve SARE via SDP by LMIs method.

1. Introduction

Stochastic linear quadratic (LQ) control problem was first studied by Wonham [1] and has become a popular research field of modern control theory, which has been extensively studied by many researchers; see, for example, [2–12]. We should point out that, in the most early literature about stochastic LQ issue, it is always assumed that the control weighting matrix 𝑅 is positive definite and the state weight matrix 𝑄 is positive semi-definite. A breakthrough belongs to [9], where a surprising fact was found that for a stochastic LQ modeled by a stochastic Itô-type differential system, even if the cost-weighting matrices 𝑄 and 𝑅 are indefinite, the original LQ optimization may still be well-posed. This finding reveals the essential difference between deterministic and stochastic systems. After that, follow-up research was carried out and a lot of important results were obtained. In [10–12], continuous-time stochastic LQ control problem with indefinite weighting matrices was studied. The authors in [10] provided necessary and sufficient conditions for the solvability of corresponding generalized differential Riccati equation (GDRE). The authors introduced LMIs whose feasibility is shown to be equivalent to the solvability of SARE and developed a computational approach to the SARE by SDP in [11]. Furthermore, stochastic indefinite LQ problems with jumps in infinite time horizon and finite time horizon were, respectively, studied in [13, 14]. Discrete-time case was also studied in [15–17]. Among these, a central issue is solving corresponding SARE. A traditional method is to consider the so-called associated Hamiltonian matrix. However, this method does not work on when 𝑅 is indefinite.

In this paper, we use SDP approach introduced in [11, 18] to discuss discrete-time indefinite stochastic LQ control problem over an infinite time horizon. Several equivalent relations between the stabilization/optimality of the LQ problem and the duality of SDP are established. We show that the stabilization is equivalent to the feasibility of the dual SDP. Furthermore, we prove that the maximal solution to SARE associated with the LQ problem can be obtained by solving the corresponding SDP. What we have obtained extend the results of [11] from continuous-time case to discrete-time case and the results of [15] from finite time horizon to infinite time horizon.

The organization of this paper is as follows. In Section 2, we formulate the discrete-time indefinite stochastic LQ problem in an infinite time horizon and present some preliminaries including some definitions, lemmas, and SDP. Section 3 is devoted to the relations between stabilization and dual SDP. In Section 4, we develop a computational approach to the SARE via SDP and characterize the optimal LQ control by the maximal solution to the SARE. Some numerical examples are presented in Section 5.

Notations 1. ℛ^𝑛: 𝑛-dimensional Euclidean space. ℛ^𝑛×𝑚: the set of all 𝑛×𝑚 matrices. 𝒮^𝑛: the set of all 𝑛×𝑛 symmetric matrices. 𝐴^: the transpose of matrix 𝐴. 𝐴≥0(𝐴>0): 𝐴 is positive semidefinite (positive definite). 𝐼: the identity matrix. 𝜎(𝐿): the spectrum set of the operator 𝐿. ℛ: the set of all real numbers. 𝒞: the set of all complex numbers. 𝒞⁻: the open left-hand side complex plane. Tr(𝑀): the trace of a square matrix 𝑀. 𝒜^adj: the adjoint mapping of 𝒜.

2. Preliminaries

2.1. Problem Statement

Consider the following discrete-time stochastic system:𝑥(𝑡+1)=𝐴𝑥𝑥(0(𝑡))=+𝑥𝐵𝑢_0,(𝑡)+[𝐶𝑥(𝑡)+𝐷𝑢(𝑡)]𝑤(𝑡),𝑡=0,1,2,…,(2.1) where 𝑥(𝑡)∈ℛ^𝑛, 𝑢(𝑡)∈ℛ^𝑚 are the system state and control input, respectively. 𝑥₀∈ℛ^𝑛 is the initial state, and 𝑤(𝑡)∈ℛ is the noise. 𝐴,𝐶∈ℛ^𝑛×𝑛 and 𝐵,𝐷∈ℛ^𝑛×𝑚 are constant matrices. {𝑤(𝑡),𝑡=0,1,2,…} is a sequence of real random variables defined on a filtered probability space (Ω,ℱ,ℱ_𝑡,𝑃) with ℱ_𝑡=𝜎{𝑤(𝑠)∶𝑠=0,1,2,…,𝑡}, which is a wide sense stationary, second-order process with 𝐸[𝑤(𝑡)]=0 and 𝐸[𝑤(𝑠)𝑤(𝑡)]=𝛿_𝑠𝑡, where 𝛿_𝑠𝑡 is the Kronecker function. 𝑢(𝑡) belongs to ℒ^2ℱ(ℛ^𝑚), the space of all ℛ^𝑚-valued, ℱ_𝑡-adapted measurable processes satisfying𝐸^∞_𝑡=0‖𝑢(𝑡)‖²<∞.(2.2) We assume that the initial state 𝑥₀ is independent of the noise 𝑤(𝑡).

We first give the following definitions.

Definition 2.1 (see [17]). The following system 𝑥(𝑡𝑥+(01))==𝑥𝐴_0,𝑥(𝑡)+𝐶𝑥(𝑡)𝑤(𝑡),𝑡=0,1,2,…,(2.3) is called asymptotically mean square stable if, for any initial state 𝑥₀, the corresponding state satisfies lim_𝑡→∞𝐸‖𝑥(𝑡)‖²=0.

Definition 2.2 (see [17]). System (2.1) is called stabilizable in the mean square sense if there exists a feedback control 𝑢(𝑡)=𝐾𝑥(𝑡) such that, for any initial state 𝑥₀, the closed-loop system 𝑥(𝑡+1)=(𝑥𝐴(0+)𝐵=𝐾𝑥)_0,𝑥(𝑡)+(𝐶+𝐷𝐾)𝑥(𝑡)𝑤(𝑡),𝑡=0,1,2,…,(2.4) is asymptotically mean square stable; that is, the corresponding state 𝑥(⋅) of (2.4) satisfies lim_𝑡→∞𝐸‖𝑥(𝑡)‖²=0, where 𝐾∈ℛ^𝑚×𝑛 is a constant matrix.
For system (2.1), we define the admissible control set 𝑈_𝑎𝑑=𝑢𝑢((𝑡𝑡))∈isℒm^2ℱeanℛsq^𝑚ua,restabilizingcontrol.(2.5) The cost function associated with system (2.1) is 𝐽𝑥₀,𝑢=^∞_𝑡=0𝐸𝑥^′(𝑡)𝑄𝑥(𝑡)+𝑢^(𝑡)𝑅𝑢(𝑡),(2.6) where 𝑄 and 𝑅 are symmetric matrices with appropriate dimensions and may be indefinite. The LQ optimal control problem is to minimize the cost functional 𝐽(𝑥₀,𝑢) over 𝑢∈𝑈_𝑎𝑑. We define the optimal value function as 𝑉𝑥₀=inf_{𝑢∈𝑈𝑎𝑑}𝐽𝑥₀,𝑢.(2.7) Since the weighting matrices 𝑄 and 𝑅 may be indefinite, the LQ problem is called an indefinite LQ control problem.

Definition 2.3. The LQ problem is called well-posed if −∞<𝑉𝑥₀<∞,∀𝑥₀∈ℛ^𝑛.(2.8) If there exists an admissible control 𝑢^∗ such that 𝑉(𝑥₀)=𝐽(𝑥₀,𝑢^∗), the LQ problem is called attainable and 𝑢^∗ is called an optimal control.
Stochastic algebraic Riccati equation (SARE) is a primary tool in solving LQ control problems. Associated with the above LQ problem, there is a discrete SARE: ℛ(𝑃)≡−𝑃+𝐴^𝑃𝐴+𝐶^𝑃𝐶+𝑄−𝑅𝐴+^𝑃𝐵𝐵_+𝐶^𝑃𝐷𝑅+𝐵^𝑃𝐵+𝐷^𝑃𝐷⁻¹𝐵^𝑃𝐴+𝐷^𝑃𝐶=0,𝑃𝐵+𝐷^𝑃𝐷>0.(2.9)

Definition 2.4. A symmetric matrix 𝑃_max is called a maximal solution to (2.9) if 𝑃_max is a solution to (2.9) and 𝑃_max≥𝑃 for any symmetric solution 𝑃 to (2.9).
Throughout this paper, we assume that system (2.1) is mean square stabilizable.

2.2. Some Definitions and Lemmas

The following definitions and lemmas will be used frequently in this paper.

Definition 2.5. For any matrix 𝑀, there exists a unique matrix 𝑀⁺, called the Moore-Penrose inverse, satisfying 𝑀𝑀⁺𝑀=𝑀,𝑀⁺𝑀𝑀⁺=𝑀⁺,𝑀𝑀⁺^=𝑀𝑀⁺,𝑀⁺𝑀^=𝑀⁺𝑀.(2.10)

Definition 2.6. Suppose that 𝒱 is a finite-dimensional vector space and 𝒮 is a space of block diagonal symmetric matrices with given dimensions. 𝒜: 𝒱→𝒮 is a linear mapping and 𝐴₀∈𝒮. Then the inequality 𝒜(𝑥)+𝐴₀≥(>)0(2.11) is called a linear matrix inequality (LMI). An LMI is called feasible if there exists at least one 𝑥∈𝒱 satisfying the above inequality and 𝑥 is called a feasible point.

Lemma 2.7 (Schur’s lemma). Let matrices 𝑀=𝑀^, 𝑁 and 𝑅=𝑅^>0 be given with appropriate dimensions. Then the following conditions are equivalent:(1)𝑀−𝑁𝑅⁻¹𝑁^≥(>)0,(2)^{𝑁𝑀𝑁′}_𝑅≥(>)0,(3)^𝑅𝑁′_𝑁𝑀≥(>)0.

Lemma 2.8 (extended Schur’s lemma). Let matrices 𝑀=𝑀^, 𝑁 and 𝑅=𝑅^ be given with appropriate dimensions. Then the following conditions are equivalent:(1)𝑀−𝑁𝑅⁺𝑁^≥0, 𝑅≥0, and 𝑁(𝐼−𝑅𝑅⁺)=0,(2)^{𝑁𝑀𝑁′}_𝑅≥0,(3)^𝑅𝑁′_𝑁𝑀≥0.

Lemma 2.9 (see [11]). For a symmetric matrix 𝑆, one has(1)𝑆⁺=(𝑆⁺)^,(2)𝑆≥0 if and only if 𝑆⁺≥0,(3)𝑆𝑆⁺=𝑆⁺𝑆.

2.3. Semidefinite Programming

Definition 2.10 (see [19]). Suppose that 𝒱 is a finite-dimensional vector space with an inner product ⟨⋅,⋅⟩_𝒱 and 𝒮 is a space of block diagonal symmetric matrices with an inner product ⟨⋅,⋅⟩_𝒮. The following optimization problem min⟨𝑐,𝑥⟩_𝒱subjectto𝐴(𝑥)=𝒜(𝑥)+𝐴₀≥0(2.12) is called a semidefinite programming (SDP). From convex duality, the dual problem associated with the SDP is defined as max−⟨𝐴₀,𝑍⟩_𝒮subjectto𝒜^adj=𝑐,𝑍≥0.(2.13) In the context of duality, we refer to the SDP (2.12) as the primal problem associated with (2.13).

Remark 2.11. Definition 2.10 is more general than Definition  6 in [11].
Let 𝑝^∗ denote the optimal value of SDP (2.12); that is, 𝑝^∗=inf⟨𝑐,𝑥⟩_𝒱∣𝐴(𝑥)≥0,(2.14) and let 𝑑^∗ denote the optimal value of the dual SDP (2.13); that is, 𝑑^∗=sup−⟨𝐴₀,𝑍⟩_𝒮∣𝑍≥0,𝒜^adj=𝑐.(2.15) Let 𝐗_opt and 𝐙_opt denote the primal and dual optimal sets; that is, 𝐙_opt𝐗_opt=𝑥∣𝐴(𝑥)≥0,⟨𝑐,𝑥⟩_𝒱=𝑝^∗,=𝑍∣𝑍≥0,𝒜^adj=𝑐,−⟨𝐴₀,𝑍⟩_𝒮=𝑑^∗.(2.16) About SDP, we have the following proposition (see [20, Theorem 3.1]).

Proposition 2.12. 𝑝^∗=𝑑^∗ if either of the following conditions holds.(1)The primal problem (2.12) is strictly feasible; that is, there is an 𝑥 with 𝐴(𝑥)>0.(2)The dual problem (2.13) is strictly feasible; that is, there is a 𝑍 with 𝑍>0 and 𝒜^adj=𝑐.

If both conditions hold, the optimal sets 𝐗_opt and 𝐙_opt are nonempty. In this case, a feasible point 𝑥 is optimal if and only if there is a feasible point 𝑍 satisfying the complementary slackness condition:𝐴(𝑥)𝑍=0.(2.17)

3. Mean Square Stabilization

The stabilization assumption of system (2.1) is basic for the study on the stochastic LQ problem for infinite horizon case. So, we will cite some equivalent conditions in verifying the stabilizability.

Lemma 3.1 (see [16, 21]). System (2.1) is mean square stabilizable if and only if one of the following conditions holds.(1)There are a matrix 𝐾 and a symmetric matrix 𝑃>0 such that −𝑃+(𝐴+𝐵𝐾)𝑃(𝐴+𝐵𝐾)^′+(𝐶+𝐷𝐾)𝑃(𝐶+𝐷𝐾)^<0.(3.1) Moreover, the stabilizing feedback control is given by 𝑢(𝑡)=𝐾𝑥(𝑡).(2)There are a matrix 𝐾 and a symmetric matrix 𝑃>0 such that −𝑃+(𝐴+𝐵𝐾)^𝑃(𝐴+𝐵𝐾)+(𝐶+𝐷𝐾)^𝑃(𝐶+𝐷𝐾)<0.(3.2) Moreover, the stabilizing feedback control is given by 𝑢(𝑡)=𝐾𝑥(𝑡).(3)For any matrix 𝑌>0, there is a matrix 𝐾 such that the following matrix equation −𝑃+(𝐴+𝐵𝐾)^𝑃(𝐴+𝐵𝐾)+(𝐶+𝐷𝐾)^𝑃(𝐶+𝐷𝐾)+𝑌=0(3.3) has a unique positive definite solution 𝑃>0. Moreover, the stabilizing feedback control is given by 𝑢(𝑡)=𝐾𝑥(𝑡).(4)For any matrix 𝑌>0, there is a matrix 𝐾 such that the following matrix equation −𝑃+(𝐴+𝐵𝐾)𝑃(𝐴+𝐵𝐾)^+(𝐶+𝐷𝐾)𝑃(𝐶+𝐷𝐾)^+𝑌=0(3.4) has a unique positive definite solution 𝑃>0. Moreover, the stabilizing feedback control is given by 𝑢(𝑡)=𝐾x(𝑡).(5)There exist matrices 𝑃>0 and 𝑈 such that the following LMI ⎡⎢⎢⎢⎢⎣𝑃𝐴_′−𝑃𝐴𝑃+𝐵𝑈𝐶𝑃+𝐷𝑈𝑃𝐶_+𝑈^𝐵^−𝑃0+𝑈^𝐷^0−𝑃⎤⎥⎥⎥⎥⎦<0(3.5) holds. Moreover, the stabilizing feedback control is given by 𝑢(𝑡)=𝑈𝑃⁻¹𝑥(𝑡).

Below, we will construct the relation between the stabilization and the dual SDP. First, we assume that the interior of the set 𝒫={𝑃∈𝒮^𝑛|ℛ(𝑃)≥0,𝑅+𝐵^𝑃𝐵+𝐷^𝑃𝐷>0} is nonempty; that is, there is a 𝑃₀∈𝒮^𝑛 such that ℛ(𝑃₀)>0 and 𝑅+𝐵^𝑃₀𝐵+𝐷^𝑃₀𝐷>0.

Consider the following SDP problem:subjemctatxo𝐴Tr(𝑃(𝑃)=)⎡⎢⎢⎢⎢⎣−𝑃+𝐴_′𝐵𝑃𝐴+𝐶^𝑃𝐶+𝑄𝐴^𝑃𝐵+𝐶^𝑃𝐷0_𝑃𝐴+0𝐷^𝑃𝐶𝑅+𝐵^𝑃𝐵0+𝐷^𝑃𝐷𝑃−0𝑃₀⎤⎥⎥⎥⎥⎦≥0.(3.6)

By the definition of SDP, we can get the dual problem of (3.6).

Theorem 3.2. The dual problem of (3.6) can be formulated as subjemctatxo⎧⎪⎪⎨⎪⎪⎩−−T𝑆r+(𝑄𝐴𝑆𝑆+𝐴_′𝑅𝑇)+Tr𝑃₀𝑊+𝐶𝑆𝐶^+𝐵𝑈𝐴^+𝐷𝑈𝐶^+𝑆𝐴𝑈𝑈^𝐵^_+𝐶𝑈^𝐷^+𝐵𝑇𝐵^+𝐷𝑇𝐷^+𝑊+𝐼=0,𝑊𝑈≥𝑇0.≥0,(3.7)

Proof. The objective of the primal problem can be rewritten as maximizing ⟨𝐼,𝑃⟩_𝒮^𝑛. Define the dual variable 𝑍∈𝒮^2𝑛+𝑚 as 𝑍=⎡⎢⎢⎢⎢⎣𝑈𝑆𝑈𝑇^𝑌^𝑌𝑊⎤⎥⎥⎥⎥⎦≥0,(3.8) where (𝑆,𝑇,𝑊,𝑈,𝑌)∈𝒮^𝑛×𝒮^𝑚×𝒮^𝑛×ℛ^𝑚×𝑛×ℛ^{𝑛×(𝑛+𝑚)}. The LMI constraint in the primal problem can be represented as 𝐴(𝑃)=𝒜(𝑃)+𝐴₀=⎡⎢⎢⎢⎢⎣−𝑃+𝐴^𝐵𝑃𝐴+𝐶^𝑃𝐶𝐴^𝑃𝐵+𝐶^𝑃𝐷0_𝑃𝐴+0𝐷^𝑃𝐶𝐵^𝑃𝐵+0𝐷^𝑃𝐷𝑃0⎤⎥⎥⎥⎥⎦+⎡⎢⎢⎢⎢⎣𝑄00𝑅00−00𝑃₀⎤⎥⎥⎥⎥⎦,(3.9) that is, 𝒜(𝑃)=⎡⎢⎢⎢⎢⎣−𝑃+𝐴^𝐵𝑃𝐴+𝐶^𝑃𝐶𝐴^𝑃𝐵+𝐶^𝑃𝐷0_𝑃𝐴+0𝐷^𝑃𝐶𝐵^𝑃𝐵+0𝐷^𝑃𝐷𝑃0⎤⎥⎥⎥⎥⎦,𝐴₀=⎡⎢⎢⎢⎢⎣𝑄00𝑅00−00𝑃₀⎤⎥⎥⎥⎥⎦.(3.10) According to the definition of adjoint mapping, we have ⟨𝒜(𝑃),𝑍⟩_𝒮^2𝑛+𝑚=⟨𝑃,𝒜^adj(𝑍)⟩_𝒮^𝑛, that is, Tr[𝒜(𝑃)𝑍]=Tr[𝑃𝒜^adj(𝑍)]. It follows 𝒜^adj(𝑍)=−𝑆+𝐴𝑆𝐴^′+𝐶𝑆𝐶^+𝐵𝑈𝐴^+𝐷𝑈𝐶^+𝐴𝑈^𝐵^+𝐶𝑈^𝐷^+𝐵𝑇𝐵^+𝐷𝑇𝐷^+𝑊. By Definition 2.10, the objective of the dual problem is to maximize −⟨𝐴₀,𝑍⟩_𝒮^2𝑛+𝑚=−Tr(𝐴₀𝑍)=−Tr(𝑄𝑆+𝑅𝑇)+Tr(𝑃₀𝑊). On the other hand, we will state that the constraints of the dual problem (2.13) are equivalent to the constraints of (3.7). Obviously, 𝒜^adj(𝑍)=−𝐼 is equivalent to the equality constraint of (3.7). Furthermore, notice that the matrix variable 𝑌 does not work on in the above formulation and therefore can be treated as zero matrix. So, the condition 𝑍≥0 is equivalent to ⎡⎢⎢⎣𝑆𝑈^′𝑈𝑇⎤⎥⎥⎦≥0,𝑊≥0.(3.11) This ends the proof.

Remark 3.3. This proof is simpler than the proof in [11] because we use a more general dual definition.

The following theorem reveals that the stabilizability of discrete stochastic system can be also regarded as a dual concept of optimality. This result is a discrete edition of Theorem  6 in [11].

Theorem 3.4. The system (2.1) is mean square stabilizable if and only if the dual problem (3.7) is strictly feasible.

Proof. First, we prove the necessary condition. Assume that system (2.1) is mean square stabilizable by the feedback 𝑢(𝑡)=𝐾𝑥(𝑡). By Lemma 3.1, there is a unique 𝑆>0 satisfying −𝑆+(𝐴+𝐵𝐾)𝑆(𝐴+𝐵𝐾)^+(𝐶+𝐷𝐾)𝑆(𝐶+𝐷𝐾)^+𝑌+𝐼=0,(3.12) where 𝑌>0 is a fixed matrix. Set 𝑈=𝐾𝑆, then 𝑈^=𝑆𝐾^. The above equality can be written as −𝑆+𝐴𝑆𝐴^+𝐴𝑈^𝐵^+𝐵𝑈𝐴^+𝐵𝑈𝑆⁻¹𝑈^𝐵^+𝐶𝑆𝐶^+𝐶𝑈^𝐷^+𝐷𝑈𝐶^+𝐷𝑈𝑆⁻¹𝑈^𝐷^+𝑌+𝐼=0.(3.13) Let 𝜀>0, 𝑇=𝜀𝐼+𝑈𝑆⁻¹𝑈^ and 𝑊=−𝜀𝐵𝐵^−𝜀𝐷𝐷^+𝑌. Obviously, 𝑇 and 𝑊 satisfy −𝑆+𝐴𝑆𝐴^+𝐴𝑈^𝐵^+𝐵𝑈𝐴^+𝐵𝑇𝐵^+𝐶𝑆𝐶^+𝐶𝑈^𝐷^+𝐷𝑈𝐶^+𝐷𝑇𝐷^+𝑊+𝐼=0.(3.14) We have 𝑊>0 for sufficiently small 𝜀>0. By Lemma 2.7, 𝑇>𝑈𝑆⁻¹𝑈^ is equivalent to ^𝑆𝑈′_𝑈𝑇>0. We conclude that the dual problem (3.7) is strictly feasible.
Next, we prove the sufficient condition. Assume that the dual problem is strictly feasible; that is, −𝑆+𝐴𝑆𝐴^+𝐶𝑆𝐶^+𝐵𝑈𝐴^+𝐷𝑈𝐶^+𝐴𝑈^𝐵^+𝐶𝑈^𝐷^+𝐵𝑇𝐵^+𝐷𝑇𝐷^+𝑊+𝐼=0, ^𝑆𝑈′_𝑈𝑇>0, 𝑊>0. It implies that there are 𝑆>0, 𝑇 and 𝑈 such that −𝑆+𝐴𝑆𝐴^+𝐴𝑈^𝐵^+𝐵𝑈𝐴^+𝐵𝑇𝑇>𝐵𝑈^+𝑆𝐶₋₁𝑆𝐶^+𝐶𝑈^𝐷^+𝐷𝑈𝐶^+𝐷𝑇𝐷^<0,𝑈^.(3.15) It follows that −𝑆+𝐴𝑆𝐴^+𝐴𝑈^𝐵^+𝐵𝑈𝐴^+𝐵𝑈𝑆⁻¹𝑈^𝐵^+𝐶𝑆𝐶^+𝐶𝑈^𝐷^+𝐷𝑈𝐶^+𝐷𝑈𝑆⁻¹𝑈^𝐷^<0.(3.16) Let 𝐾=𝑈𝑆⁻¹. The above inequality is equivalent to −𝑆+(𝐴+𝐵𝐾)𝑆(𝐴+𝐵𝐾)^+(𝐶+𝐷𝐾)𝑆(𝐶+𝐷𝐾)^<0.(3.17) By Lemma 3.1, system (2.1) is mean square stabilizable.

4. Solutions to SARE and SDP

The following theorem will state the existence of the solution of the SARE (2.9) via SDP (3.6).

Theorem 4.1. The optimal set of (3.6) is nonempty, and any optimal solution 𝑃^∗ must satisfy the SARE (2.9).

Proof. Since system (2.1) is mean square stabilizable, by Theorem 3.4, (3.7) is strictly feasible. Equation (3.6) is strictly feasible because 𝑃₀ is a interior point of 𝒫. By Proposition 2.12, (3.6) is nonempty and 𝑃^∗ satisfies 𝐴(𝑃^∗)𝑍=0; that is, ⎡⎢⎢⎢⎢⎣−𝑃^∗+𝐵𝐴^_𝑃^∗𝐴+𝐶^𝑃^∗𝐶+𝑄𝐴^𝑃^∗𝐵+𝐶^𝑃^∗𝐷𝑃^∗𝐴+𝐷^𝑃^∗𝐶𝐵^𝑃^∗𝐵+𝐷^𝑃^∗𝐷+𝑅00𝑃^∗−𝑃₀⎤⎥⎥⎥⎥⎦⎡⎢⎢⎢⎢⎣𝑆𝑈^𝑈𝑇00𝑊⎤⎥⎥⎥⎥⎦=0.(4.1) From the above equality, we have the following equalities: −−𝑃𝑃^∗_∗+𝐴^𝑃^∗𝐴+𝐶^𝑃^∗𝐶+𝑄𝑆+𝐴^𝑃^∗𝐵+𝐶^𝑃^∗𝐷𝑈=0,(4.2)𝐵+𝐴^𝑃^∗𝐴+𝐶^𝑃^∗𝐶+𝑄𝑈^+𝐴^𝑃^∗𝐵+𝐶^𝑃^∗𝐷𝑇=0,(4.3)_𝐵𝑃^∗𝐴+𝐷^𝑃^∗𝐶𝑆+𝐵^𝑃^∗𝐵+𝐷^𝑃^∗𝐷+𝑅𝑈=0,(4.4)_𝑃^∗𝐴+𝐷^𝑃^∗𝐶𝑈𝑃^+𝐵^𝑃^∗𝐵+𝐷^𝑃^∗𝐷+𝑅𝑇=0,(4.5)_∗−𝑃₀𝑊=0.(4.6) Moreover, 𝑅+𝐷^𝑃^∗𝐷+𝐵^𝑃^∗𝐵>0 because of 𝑅+𝐷^𝑃₀𝐷+𝐵^𝑃₀𝐵>0 and 𝑃^∗≥𝑃₀. Then by (4.4), 𝑈=−(𝐵^𝑃^∗𝐵+𝐷^𝑃^∗𝐷+𝑅)⁻¹(𝐵^𝑃^∗𝐴+𝐷^𝑃^∗𝐶)𝑆. Substituting it into (4.2) yields ℛ(𝑃^∗)𝑆=0. Remember that 𝑆, 𝑇, 𝑈, 𝑊 satisfy the equality constraint in (3.7). Multiplying both sides by ℛ(𝑃^∗), we have ℛ𝑃^∗𝐴𝑆𝐴^+𝐶𝑆𝐶^+𝐵𝑈𝐴^+𝐷𝑈𝐶^+𝐴𝑈^𝐵^+𝐶𝑈^𝐷^+𝐵𝑇𝐵^+𝐷𝑇𝐷^+𝑊+𝐼ℛ𝑃^∗=0.(4.7) Considering ^𝑆𝑈′_𝑈𝑇≥0, it follows from Lemma 2.8 that 𝑇≥𝑈𝑆⁺𝑈^ and 𝑈=𝑈𝑆𝑆⁺. By Lemma 2.9, 𝑆⁺≥0 and 𝑆𝑆⁺=𝑆⁺𝑆. So we have ℛ𝑃=^∗ℛ𝐴𝑃𝑆_∗𝐴^+𝐶𝑆𝐶^+𝐵𝑈𝐴^+𝐷𝑈𝐶^+𝐴𝑈^𝐵^+𝐶𝑈^𝐷^+𝐵𝑈𝑆⁺𝑈^𝐵^+𝐷𝑈𝑆⁺𝑈^𝐷^ℛ𝑃^∗(𝐶𝑆+𝐷𝑈)𝑆⁺𝑆𝐶^+𝑈^𝐷^+(𝐴𝑆+𝐵𝑈)𝑆⁺𝑆𝐴^+𝑈^𝐵^ℛ𝑃^∗≤0.(4.8) Then it follows that ℛ(𝑃^∗)ℛ(𝑃^∗)≤0. It yields ℛ(𝑃^∗)=0 due to ℛ(𝑃^∗)≥0.

The following theorem shows that any optimal solution of the primal SDP results in a stabilizing control for LQ problem.

Theorem 4.2. Let 𝑃^∗ be an optimal solution to the SDP (3.6). Then the feedback control 𝑢(𝑡)=−(𝑅+𝐵^𝑃^∗𝐵+𝐷^𝑃^∗𝐷)⁻¹(𝐵^𝑃^∗𝐴+𝐷^𝑃^∗𝐶)𝑥(𝑡) is mean square stabilizing for system (2.1).

Proof. Optimal dual variables 𝑆, 𝑇, 𝑈, 𝑊 satisfy (4.2)–(4.6). 𝑈=−(𝐵^𝑃^∗𝐵+𝐷^𝑃^∗𝐷+𝑅)⁻¹(𝐵^𝑃^∗𝐴+𝐷^𝑃^∗𝐶)𝑆. Now we show 𝑆>0. Let 𝑆𝑥=0, 𝑥∈ℛ^𝑛. The constraints in (3.7) imply 𝑥^𝐴𝑆𝐴^+𝐶𝑆𝐶^+𝐵𝑈𝐴^+𝐷𝑈𝐶^+𝐴𝑈^𝐵^+𝐶𝑈^𝐷^+𝐵𝑈𝑆⁺𝑈^𝐵^+𝐷𝑈𝑆⁺𝑈^𝐷^+𝑊+𝐼𝑥≤0.(4.9) Similar to the proof of Theorem 4.1, we have 𝑥=0. We conclude that 𝑆>0 from 𝑆≥0. Again by the equality constraint in (3.7), we have −𝑆+𝐴𝑆𝐴^+𝐶𝑆𝐶^+𝐵𝑈𝐴^+𝐷𝑈𝐶^+𝐴𝑈^𝐵^+𝐶𝑈^𝐷^+𝐵𝑈𝑆⁻¹𝑈^𝐵^+𝐷𝑈𝑆⁻¹𝑈^𝐷^=−𝑆+𝐴+𝐵𝑈𝑆⁻¹𝑆𝐴+𝐵𝑈𝑆⁻¹^+𝐶+𝐷𝑈𝑆⁻¹𝑆𝐶+𝐷𝑈𝑆⁻¹^<0.(4.10) By Lemma 3.1, the above inequality is equivalent to the mean square stabilizability of system (2.1) with 𝑢(𝑡)=𝑈𝑆⁻¹𝑥(𝑡)=−(𝑅+𝐵^𝑃^∗𝐵+𝐷^𝑃^∗𝐷)⁻¹(𝐵^𝑃^∗𝐴+𝐷^𝑃^∗𝐶)𝑥(𝑡). This ends the proof.

Theorem 4.3. There is a unique optimal solution to (3.6), which is the maximal solution to SARE (2.9).

Proof. The proof is similar to Theorem  9 in [11] and is omitted.

Theorem 4.4. Assume that 𝒫 is nonempty, then SARE (2.9) has a maximal solution 𝑃_max, which is the unique optimal solution to SDP: subjemctatxo𝐴Tr(𝑃(𝑃)=)⎡⎢⎢⎣−𝑃+𝐴_′𝐵𝑃𝐴+𝐶^𝑃𝐶+𝑄𝐴^𝑃𝐵+𝐶^𝑃𝐷_𝑅+𝐵_𝑃𝐴+𝐷^𝑃𝐶𝑅+𝐵^𝑃𝐵+𝐷^𝑃𝐷⎤⎥⎥⎦𝑃𝐵+𝐷^𝑃D>0.≥0,(4.11)

Proof. The first assertion is Theorem  4 in [16]. As to the second part, we proceed as follows. By Lemma 2.7, 𝑃_max satisfies the constraints in (4.11). 𝑃_max is an optimal solution to (4.11) due to the maximality. Next we prove the uniqueness. Assume that 𝑃^′_max is another optimal solution to (4.11). Then we have Tr(𝑃_max)=Tr(𝑃^max). According to Definition 2.4, 𝑃_max≥𝑃^max. This yields 𝑃_max=𝑃^max. Hence, the proof of the theorem is completed.

Remark 4.5. Here we drop the assumption that the interior of 𝒫 is nonempty.

Remark 4.6. Theorem 4.4 presents that the maximal solution to SARE (2.9) can be obtained by solving SDP (4.11). The result provides us a computational approach to the SARE. Furthermore, as shown in [16], the relationship between the LQ value function and the maximal solution to SARE (2.9) can be established; that is, assuming that 𝒫 is nonempty, then the value function 𝑉(𝑥₀)=𝑥^0𝑃_max𝑥₀ and the optimal control can be expressed as 𝑢(𝑡)=−(𝑅+𝐵^𝑃_max𝐵+𝐷^𝑃_max𝐷)⁻¹(𝐵^𝑃_max𝐴+𝐷^𝑃_max𝐶)𝑥(𝑡).
The above results represent SARE (2.9) may exist a solution even if 𝑅 is indefinite (even negative definite). To describe the allowable negative degree, we give the following definition to solvability margin.

Definition 4.7 (see [11]). The solvability margin 𝑟^∗ is defined as the largest nonnegative scalar 𝑟≥0 such that (2.9) has a solution for any 𝑅>−𝑟^∗𝐼.

By Theorem 4.4, the following conclusion is obvious.

Theorem 4.8. The solvability margin 𝑟^∗ can be obtained by solving the following SDP: subjemctatxo𝑟⎧⎪⎨⎪⎩−𝑃+𝐴_′𝐵𝑃𝐴+𝐶^𝑃𝐶+𝑄𝐴^𝑃𝐵+𝐶^𝑃𝐷_𝐵𝑃𝐴+𝐷^𝑃𝐶𝐵^𝑃𝐵+𝐷^𝑃𝐷−𝑟𝐼≥0,_𝑟>𝑃𝐵0.+𝐷^𝑃𝐷−𝑟𝐼>0,(4.12)

5. Numerical Examples

Consider the system (2.1) with 𝐴, 𝐵, 𝐶, 𝐷 as follows:𝐴𝐶==⎡⎢⎢⎢⎢⎣−−⎡⎢⎢⎢⎢⎣0−01.001..046..217.361572258345666687−21−−1..221.001.891..451708683793586234−100010..13....1000821563973967239386⎤⎥⎥⎥⎥⎦⎤⎥⎥⎥⎥⎦,,𝐷𝐵==⎡⎢⎢⎢⎢⎣⎡⎢⎢⎢⎢⎣−−0000...00259.1..74589286710759545078−00−0..0091.09..6506.7479930961726354⎤⎥⎥⎥⎥⎦4.⎤⎥⎥⎥⎥⎦,(5.1)

5.1. Mean Square Stabilizability

In order to test the mean square stabilizability of system (2.1), we only need to check weather or not LMI (3.5) is feasible by Theorem 3.2. Making use of LMI feasp solver [22], we find matrices 𝑃 and 𝑈 satisfying (3.5):𝑃=⎡⎢⎢⎢⎢⎣402...820217005357−012..22.2182018527−252..03.2782588757⎤⎥⎥⎥⎥⎦,𝑈=⎡⎢⎢⎣−45.5.4419895−00.9.9422268−01.5.3717103⎤⎥⎥⎦,(5.2) and the stabilizing control 𝑢(𝑡)=𝑈𝑃⁻¹𝑥(𝑡); that is,𝑢(𝑡)=⎡⎢⎢⎣−00.9.6667096−02.4.1017196−−00..08967203⎤⎥⎥⎦𝑥(𝑡).(5.3)

5.2. Solutions of SARE

Let 𝑄=𝐼 in SARE (2.9). Below, we solve SARE (2.9) via the SDP (4.11) (LMI mincx solver [22]).

Case 1. 𝑅 is positive definite. Choose 𝑅=²¹¹², and we obtain the maximal solution to (2.9): 𝑃_max=⎡⎢⎢⎢⎢⎣−−414408.02..87528770414−311680020..64.5957474224−1984428.6..48152077321⎤⎥⎥⎥⎥⎦.(5.4)

Case 2. 𝑅 is indefinite. Choose 𝑅=^0.20.30.30.2, and we obtain the maximal solution to (2.9): 𝑃_max=⎡⎢⎢⎢⎢⎣−−141709.7..01999770530−1634750..6.20719987093−631759..65.1719910070⎤⎥⎥⎥⎥⎦.(5.5)

Case 3. 𝑅 is negative definite. First we can get the solvability margin 𝑟^∗=0.4936 by solving SDP (4.12) (LMI gevp solver [22]). Hence (2.9) has a maximal solution when 𝑅>−0.4936𝐼. Choose 𝑅=−0.4𝐼, and we obtain the maximal solution to (2.9): 𝑃_max=⎡⎢⎢⎢⎢⎣−−263962.6..72284893371−21316476...097708969257−1613172..3.92304063151⎤⎥⎥⎥⎥⎦.(5.6)

6. Conclusion

In this paper, we use the SDP approach to the study of discrete-time indefinite stochastic LQ control. It was shown that the mean square stabilization of system (2.1) is equivalent to the strict feasibility of the SDP (3.7). In addition, the relation between the optimal solution of (3.6) and the maximal solution of SARE (2.9) has been established. What we have obtained can be viewed as a discrete-time version of [11]. Of course, there are many open problems to be solved. For example, 𝑅+𝐵^𝑃𝐵+𝐷^𝑃𝐷>0 is a basic assumption in this paper. A natural question is whether or not we can weaken it to 𝑅+𝐵^𝑃𝐵+𝐷^𝑃𝐷≥0. This problem merits further study.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (61174078), Specialized Research Fund for the Doctoral Program of Higher Education (20103718110006), and Key Project of Natural Science Foundation of Shandong Province (ZR2009GZ001).