About this Journal Submit a Manuscript Table of Contents
Journal of Applied Mathematics
Volume 2012 (2012), Article ID 638762, 14 pages
http://dx.doi.org/10.1155/2012/638762
Research Article

Discrete-Time Indefinite Stochastic LQ Control via SDP and LMI Methods

College of Information and Electrical Engineering, Shandong University of Science and Technology, Qingdao 266510, China

Received 10 August 2011; Accepted 4 December 2011

Academic Editor: Oluwole D. Makinde

Copyright © 2012 Shaowei Zhou and Weihai Zhang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

This paper studies a discrete-time stochastic LQ problem over an infinite time horizon with state-and control-dependent noises, whereas the weighting matrices in the cost function are allowed to be indefinite. We mainly use semidefinite programming (SDP) and its duality to treat corresponding problems. Several relations among stability, SDP complementary duality, the existence of the solution to stochastic algebraic Riccati equation (SARE), and the optimality of LQ problem are established. We can test mean square stabilizability and solve SARE via SDP by LMIs method.

1. Introduction

Stochastic linear quadratic (LQ) control problem was first studied by Wonham [1] and has become a popular research field of modern control theory, which has been extensively studied by many researchers; see, for example, [212]. We should point out that, in the most early literature about stochastic LQ issue, it is always assumed that the control weighting matrix 𝑅 is positive definite and the state weight matrix 𝑄 is positive semi-definite. A breakthrough belongs to [9], where a surprising fact was found that for a stochastic LQ modeled by a stochastic Itô-type differential system, even if the cost-weighting matrices 𝑄 and 𝑅 are indefinite, the original LQ optimization may still be well-posed. This finding reveals the essential difference between deterministic and stochastic systems. After that, follow-up research was carried out and a lot of important results were obtained. In [1012], continuous-time stochastic LQ control problem with indefinite weighting matrices was studied. The authors in [10] provided necessary and sufficient conditions for the solvability of corresponding generalized differential Riccati equation (GDRE). The authors introduced LMIs whose feasibility is shown to be equivalent to the solvability of SARE and developed a computational approach to the SARE by SDP in [11]. Furthermore, stochastic indefinite LQ problems with jumps in infinite time horizon and finite time horizon were, respectively, studied in [13, 14]. Discrete-time case was also studied in [1517]. Among these, a central issue is solving corresponding SARE. A traditional method is to consider the so-called associated Hamiltonian matrix. However, this method does not work on when 𝑅 is indefinite.

In this paper, we use SDP approach introduced in [11, 18] to discuss discrete-time indefinite stochastic LQ control problem over an infinite time horizon. Several equivalent relations between the stabilization/optimality of the LQ problem and the duality of SDP are established. We show that the stabilization is equivalent to the feasibility of the dual SDP. Furthermore, we prove that the maximal solution to SARE associated with the LQ problem can be obtained by solving the corresponding SDP. What we have obtained extend the results of [11] from continuous-time case to discrete-time case and the results of [15] from finite time horizon to infinite time horizon.

The organization of this paper is as follows. In Section 2, we formulate the discrete-time indefinite stochastic LQ problem in an infinite time horizon and present some preliminaries including some definitions, lemmas, and SDP. Section 3 is devoted to the relations between stabilization and dual SDP. In Section 4, we develop a computational approach to the SARE via SDP and characterize the optimal LQ control by the maximal solution to the SARE. Some numerical examples are presented in Section 5.

Notations 1. 𝑛: 𝑛-dimensional Euclidean space. 𝑛×𝑚: the set of all 𝑛×𝑚 matrices. 𝒮𝑛: the set of all 𝑛×𝑛 symmetric matrices. 𝐴: the transpose of matrix 𝐴. 𝐴0(𝐴>0): 𝐴 is positive semidefinite (positive definite). 𝐼: the identity matrix. 𝜎(𝐿): the spectrum set of the operator 𝐿. : the set of all real numbers. 𝒞: the set of all complex numbers. 𝒞: the open left-hand side complex plane. Tr(𝑀): the trace of a square matrix 𝑀. 𝒜adj: the adjoint mapping of 𝒜.

2. Preliminaries

2.1. Problem Statement

Consider the following discrete-time stochastic system:[]𝑥𝑥(𝑡+1)=𝐴𝑥(𝑡)+𝐵𝑢(𝑡)+𝐶𝑥(𝑡)+𝐷𝑢(𝑡)𝑤(𝑡),(0)=𝑥0,𝑡=0,1,2,,(2.1) where 𝑥(𝑡)𝑛, 𝑢(𝑡)𝑚 are the system state and control input, respectively. 𝑥0𝑛 is the initial state, and 𝑤(𝑡) is the noise. 𝐴,𝐶𝑛×𝑛 and 𝐵,𝐷𝑛×𝑚 are constant matrices. {𝑤(𝑡),𝑡=0,1,2,} is a sequence of real random variables defined on a filtered probability space (Ω,,𝑡,𝑃) with 𝑡=𝜎{𝑤(𝑠)𝑠=0,1,2,,𝑡}, which is a wide sense stationary, second-order process with 𝐸[𝑤(𝑡)]=0 and 𝐸[𝑤(𝑠)𝑤(𝑡)]=𝛿𝑠𝑡, where 𝛿𝑠𝑡 is the Kronecker function. 𝑢(𝑡) belongs to 2(𝑚), the space of all 𝑚-valued, 𝑡-adapted measurable processes satisfying𝐸𝑡=0𝑢(𝑡)2<.(2.2) We assume that the initial state 𝑥0 is independent of the noise 𝑤(𝑡).

We first give the following definitions.

Definition 2.1 (see [17]). The following system 𝑥𝑥(𝑡+1)=𝐴𝑥(𝑡)+𝐶𝑥(𝑡)𝑤(𝑡),(0)=𝑥0,𝑡=0,1,2,,(2.3) is called asymptotically mean square stable if, for any initial state 𝑥0, the corresponding state satisfies lim𝑡𝐸𝑥(𝑡)2=0.

Definition 2.2 (see [17]). System (2.1) is called stabilizable in the mean square sense if there exists a feedback control 𝑢(𝑡)=𝐾𝑥(𝑡) such that, for any initial state 𝑥0, the closed-loop system 𝑥𝑥(𝑡+1)=(𝐴+𝐵𝐾)𝑥(𝑡)+(𝐶+𝐷𝐾)𝑥(𝑡)𝑤(𝑡),(0)=𝑥0,𝑡=0,1,2,,(2.4) is asymptotically mean square stable; that is, the corresponding state 𝑥() of (2.4) satisfies lim𝑡𝐸𝑥(𝑡)2=0, where 𝐾𝑚×𝑛 is a constant matrix.
For system (2.1), we define the admissible control set 𝑈𝑎𝑑=𝑢(𝑡)2𝑚,𝑢(𝑡)ismeansquarestabilizingcontrol.(2.5) The cost function associated with system (2.1) is 𝐽𝑥0=,𝑢𝑡=0𝐸𝑥(𝑡)𝑄𝑥(𝑡)+𝑢(,𝑡)𝑅𝑢(𝑡)(2.6) where 𝑄 and 𝑅 are symmetric matrices with appropriate dimensions and may be indefinite. The LQ optimal control problem is to minimize the cost functional 𝐽(𝑥0,𝑢) over 𝑢𝑈𝑎𝑑. We define the optimal value function as 𝑉𝑥0=inf𝑢𝑈𝑎𝑑𝐽𝑥0.,𝑢(2.7) Since the weighting matrices 𝑄 and 𝑅 may be indefinite, the LQ problem is called an indefinite LQ control problem.

Definition 2.3. The LQ problem is called well-posed if 𝑥<𝑉0<,𝑥0𝑛.(2.8) If there exists an admissible control 𝑢 such that 𝑉(𝑥0)=𝐽(𝑥0,𝑢), the LQ problem is called attainable and 𝑢 is called an optimal control.
Stochastic algebraic Riccati equation (SARE) is a primary tool in solving LQ control problems. Associated with the above LQ problem, there is a discrete SARE: (𝑃)𝑃+𝐴𝑃𝐴+𝐶𝐴𝑃𝐶+𝑄𝑃𝐵+𝐶𝑃𝐷𝑅+𝐵𝑃𝐵+𝐷𝑃𝐷1𝐵𝑃𝐴+𝐷𝑃𝐶=0,𝑅+𝐵𝑃𝐵+𝐷𝑃𝐷>0.(2.9)

Definition 2.4. A symmetric matrix 𝑃max is called a maximal solution to (2.9) if 𝑃max is a solution to (2.9) and 𝑃max𝑃 for any symmetric solution 𝑃 to (2.9).
Throughout this paper, we assume that system (2.1) is mean square stabilizable.

2.2. Some Definitions and Lemmas

The following definitions and lemmas will be used frequently in this paper.

Definition 2.5. For any matrix 𝑀, there exists a unique matrix 𝑀+, called the Moore-Penrose inverse, satisfying 𝑀𝑀+𝑀=𝑀,𝑀+𝑀𝑀+=𝑀+,𝑀𝑀+=𝑀𝑀+,𝑀+𝑀=𝑀+𝑀.(2.10)

Definition 2.6. Suppose that 𝒱 is a finite-dimensional vector space and 𝒮 is a space of block diagonal symmetric matrices with given dimensions. 𝒜: 𝒱𝒮 is a linear mapping and 𝐴0𝒮. Then the inequality 𝒜(𝑥)+𝐴0(>)0(2.11) is called a linear matrix inequality (LMI). An LMI is called feasible if there exists at least one 𝑥𝒱 satisfying the above inequality and 𝑥 is called a feasible point.

Lemma 2.7 (Schur’s lemma). Let matrices 𝑀=𝑀, 𝑁 and 𝑅=𝑅>0 be given with appropriate dimensions. Then the following conditions are equivalent:(1)𝑀𝑁𝑅1𝑁(>)0,(2)𝑁𝑀𝑁𝑅(>)0,(3)𝑅𝑁𝑁𝑀(>)0.

Lemma 2.8 (extended Schur’s lemma). Let matrices 𝑀=𝑀, 𝑁 and 𝑅=𝑅 be given with appropriate dimensions. Then the following conditions are equivalent:(1)𝑀𝑁𝑅+𝑁0, 𝑅0, and 𝑁(𝐼𝑅𝑅+)=0,(2)𝑁𝑀𝑁𝑅0,(3)𝑅𝑁𝑁𝑀0.

Lemma 2.9 (see [11]). For a symmetric matrix 𝑆, one has(1)𝑆+=(𝑆+),(2)𝑆0 if and only if 𝑆+0,(3)𝑆𝑆+=𝑆+𝑆.

2.3. Semidefinite Programming

Definition 2.10 (see [19]). Suppose that 𝒱 is a finite-dimensional vector space with an inner product ,𝒱 and 𝒮 is a space of block diagonal symmetric matrices with an inner product ,𝒮. The following optimization problem min𝑐,𝑥𝒱subjectto𝐴(𝑥)=𝒜(𝑥)+𝐴00(2.12) is called a semidefinite programming (SDP). From convex duality, the dual problem associated with the SDP is defined as max𝐴0,𝑍𝒮subjectto𝒜adj=𝑐,𝑍0.(2.13) In the context of duality, we refer to the SDP (2.12) as the primal problem associated with (2.13).

Remark 2.11. Definition 2.10 is more general than Definition  6 in [11].
Let 𝑝 denote the optimal value of SDP (2.12); that is, 𝑝=inf𝑐,𝑥𝒱𝐴(𝑥)0,(2.14) and let 𝑑 denote the optimal value of the dual SDP (2.13); that is, 𝑑=sup𝐴0,𝑍𝒮𝑍0,𝒜adj.=𝑐(2.15) Let 𝐗opt and 𝐙opt denote the primal and dual optimal sets; that is, 𝐗opt=𝑥𝐴(𝑥)0,𝑐,𝑥𝒱=𝑝,𝐙opt=𝑍𝑍0,𝒜adj=𝑐,𝐴0,𝑍𝒮=𝑑.(2.16) About SDP, we have the following proposition (see [20, Theorem 3.1]).

Proposition 2.12. 𝑝=𝑑 if either of the following conditions holds.(1)The primal problem (2.12) is strictly feasible; that is, there is an 𝑥 with 𝐴(𝑥)>0.(2)The dual problem (2.13) is strictly feasible; that is, there is a 𝑍 with 𝑍>0 and 𝒜adj=𝑐.

If both conditions hold, the optimal sets 𝐗opt and 𝐙opt are nonempty. In this case, a feasible point 𝑥 is optimal if and only if there is a feasible point 𝑍 satisfying the complementary slackness condition:𝐴(𝑥)𝑍=0.(2.17)

3. Mean Square Stabilization

The stabilization assumption of system (2.1) is basic for the study on the stochastic LQ problem for infinite horizon case. So, we will cite some equivalent conditions in verifying the stabilizability.

Lemma 3.1 (see [16, 21]). System (2.1) is mean square stabilizable if and only if one of the following conditions holds.(1)There are a matrix 𝐾 and a symmetric matrix 𝑃>0 such that 𝑃+(𝐴+𝐵𝐾)𝑃(𝐴+𝐵𝐾)+(𝐶+𝐷𝐾)𝑃(𝐶+𝐷𝐾)<0.(3.1) Moreover, the stabilizing feedback control is given by 𝑢(𝑡)=𝐾𝑥(𝑡).(2)There are a matrix 𝐾 and a symmetric matrix 𝑃>0 such that 𝑃+(𝐴+𝐵𝐾)𝑃(𝐴+𝐵𝐾)+(𝐶+𝐷𝐾)𝑃(𝐶+𝐷𝐾)<0.(3.2) Moreover, the stabilizing feedback control is given by 𝑢(𝑡)=𝐾𝑥(𝑡).(3)For any matrix 𝑌>0, there is a matrix 𝐾 such that the following matrix equation 𝑃+(𝐴+𝐵𝐾)𝑃(𝐴+𝐵𝐾)+(𝐶+𝐷𝐾)𝑃(𝐶+𝐷𝐾)+𝑌=0(3.3) has a unique positive definite solution 𝑃>0. Moreover, the stabilizing feedback control is given by 𝑢(𝑡)=𝐾𝑥(𝑡).(4)For any matrix 𝑌>0, there is a matrix 𝐾 such that the following matrix equation 𝑃+(𝐴+𝐵𝐾)𝑃(𝐴+𝐵𝐾)+(𝐶+𝐷𝐾)𝑃(𝐶+𝐷𝐾)+𝑌=0(3.4) has a unique positive definite solution 𝑃>0. Moreover, the stabilizing feedback control is given by 𝑢(𝑡)=𝐾x(𝑡).(5)There exist matrices 𝑃>0 and 𝑈 such that the following LMI 𝑃𝐴𝑃+𝐵𝑈𝐶𝑃+𝐷𝑈𝑃𝐴+𝑈𝐵𝑃0𝑃𝐶+𝑈𝐷0𝑃<0(3.5) holds. Moreover, the stabilizing feedback control is given by 𝑢(𝑡)=𝑈𝑃1𝑥(𝑡).

Below, we will construct the relation between the stabilization and the dual SDP. First, we assume that the interior of the set 𝒫={𝑃𝒮𝑛|(𝑃)0,𝑅+𝐵𝑃𝐵+𝐷𝑃𝐷>0} is nonempty; that is, there is a 𝑃0𝒮𝑛 such that (𝑃0)>0 and 𝑅+𝐵𝑃0𝐵+𝐷𝑃0𝐷>0.

Consider the following SDP problem:maxTr(𝑃)subjectto𝐴(𝑃)=𝑃+𝐴𝑃𝐴+𝐶𝑃𝐶+𝑄𝐴𝑃𝐵+𝐶𝐵𝑃𝐷0𝑃𝐴+𝐷𝑃𝐶𝑅+𝐵𝑃𝐵+𝐷𝑃𝐷000𝑃𝑃00.(3.6)

By the definition of SDP, we can get the dual problem of (3.6).

Theorem 3.2. The dual problem of (3.6) can be formulated as 𝑃maxTr(𝑄𝑆+𝑅𝑇)+Tr0𝑊subjectto𝑆+𝐴𝑆𝐴+𝐶𝑆𝐶+𝐵𝑈𝐴+𝐷𝑈𝐶+𝐴𝑈𝐵+𝐶𝑈𝐷+𝐵𝑇𝐵+𝐷𝑇𝐷+𝑊+𝐼=0,𝑆𝑈𝑈𝑇0,𝑊0.(3.7)

Proof. The objective of the primal problem can be rewritten as maximizing 𝐼,𝑃𝒮𝑛. Define the dual variable 𝑍𝒮2𝑛+𝑚 as 𝑍=𝑆𝑈𝑈𝑇𝑌𝑌𝑊0,(3.8) where (𝑆,𝑇,𝑊,𝑈,𝑌)𝒮𝑛×𝒮𝑚×𝒮𝑛×𝑚×𝑛×𝑛×(𝑛+𝑚). The LMI constraint in the primal problem can be represented as 𝐴(𝑃)=𝒜(𝑃)+𝐴0=𝑃+𝐴𝑃𝐴+𝐶𝑃𝐶𝐴𝑃𝐵+𝐶𝐵𝑃𝐷0𝑃𝐴+𝐷𝑃𝐶𝐵𝑃𝐵+𝐷+𝑃𝐷000𝑃𝑄000𝑅000𝑃0,(3.9) that is, 𝒜(𝑃)=𝑃+𝐴𝑃𝐴+𝐶𝑃𝐶𝐴𝑃𝐵+𝐶𝐵𝑃𝐷0𝑃𝐴+𝐷𝑃𝐶𝐵𝑃𝐵+𝐷𝑃𝐷000𝑃,𝐴0=𝑄000𝑅000𝑃0.(3.10) According to the definition of adjoint mapping, we have 𝒜(𝑃),𝑍𝒮2𝑛+𝑚=𝑃,𝒜adj(𝑍)𝒮𝑛, that is, Tr[𝒜(𝑃)𝑍]=Tr[𝑃𝒜adj(𝑍)]. It follows 𝒜adj(𝑍)=𝑆+𝐴𝑆𝐴+𝐶𝑆𝐶+𝐵𝑈𝐴+𝐷𝑈𝐶+𝐴𝑈𝐵+𝐶𝑈𝐷+𝐵𝑇𝐵+𝐷𝑇𝐷+𝑊. By Definition 2.10, the objective of the dual problem is to maximize 𝐴0,𝑍𝒮2𝑛+𝑚=Tr(𝐴0𝑍)=Tr(𝑄𝑆+𝑅𝑇)+Tr(𝑃0𝑊). On the other hand, we will state that the constraints of the dual problem (2.13) are equivalent to the constraints of (3.7). Obviously, 𝒜adj(𝑍)=𝐼 is equivalent to the equality constraint of (3.7). Furthermore, notice that the matrix variable 𝑌 does not work on in the above formulation and therefore can be treated as zero matrix. So, the condition 𝑍0 is equivalent to 𝑆𝑈𝑈𝑇0,𝑊0.(3.11) This ends the proof.

Remark 3.3. This proof is simpler than the proof in [11] because we use a more general dual definition.

The following theorem reveals that the stabilizability of discrete stochastic system can be also regarded as a dual concept of optimality. This result is a discrete edition of Theorem  6 in [11].

Theorem 3.4. The system (2.1) is mean square stabilizable if and only if the dual problem (3.7) is strictly feasible.

Proof. First, we prove the necessary condition. Assume that system (2.1) is mean square stabilizable by the feedback 𝑢(𝑡)=𝐾𝑥(𝑡). By Lemma 3.1, there is a unique 𝑆>0 satisfying 𝑆+(𝐴+𝐵𝐾)𝑆(𝐴+𝐵𝐾)+(𝐶+𝐷𝐾)𝑆(𝐶+𝐷𝐾)+𝑌+𝐼=0,(3.12) where 𝑌>0 is a fixed matrix. Set 𝑈=𝐾𝑆, then 𝑈=𝑆𝐾. The above equality can be written as 𝑆+𝐴𝑆𝐴+𝐴𝑈𝐵+𝐵𝑈𝐴+𝐵𝑈𝑆1𝑈𝐵+𝐶𝑆𝐶+𝐶𝑈𝐷+𝐷𝑈𝐶+𝐷𝑈𝑆1𝑈𝐷+𝑌+𝐼=0.(3.13) Let 𝜀>0, 𝑇=𝜀𝐼+𝑈𝑆1𝑈 and 𝑊=𝜀𝐵𝐵𝜀𝐷𝐷+𝑌. Obviously, 𝑇 and 𝑊 satisfy 𝑆+𝐴𝑆𝐴+𝐴𝑈𝐵+𝐵𝑈𝐴+𝐵𝑇𝐵+𝐶𝑆𝐶+𝐶𝑈𝐷+𝐷𝑈𝐶+𝐷𝑇𝐷+𝑊+𝐼=0.(3.14) We have 𝑊>0 for sufficiently small 𝜀>0. By Lemma 2.7, 𝑇>𝑈𝑆1𝑈 is equivalent to 𝑆𝑈𝑈𝑇>0. We conclude that the dual problem (3.7) is strictly feasible.
Next, we prove the sufficient condition. Assume that the dual problem is strictly feasible; that is, 𝑆+𝐴𝑆𝐴+𝐶𝑆𝐶+𝐵𝑈𝐴+𝐷𝑈𝐶+𝐴𝑈𝐵+𝐶𝑈𝐷+𝐵𝑇𝐵+𝐷𝑇𝐷+𝑊+𝐼=0, 𝑆𝑈𝑈𝑇>0, 𝑊>0. It implies that there are 𝑆>0, 𝑇 and 𝑈 such that 𝑆+𝐴𝑆𝐴+𝐴𝑈𝐵+𝐵𝑈𝐴+𝐵𝑇𝐵+𝐶𝑆𝐶+𝐶𝑈𝐷+𝐷𝑈𝐶+𝐷𝑇𝐷<0,𝑇>𝑈𝑆1𝑈.(3.15) It follows that 𝑆+𝐴𝑆𝐴+𝐴𝑈𝐵+𝐵𝑈𝐴+𝐵𝑈𝑆1𝑈𝐵+𝐶𝑆𝐶+𝐶𝑈𝐷+𝐷𝑈𝐶+𝐷𝑈𝑆1𝑈𝐷<0.(3.16) Let 𝐾=𝑈𝑆1. The above inequality is equivalent to 𝑆+(𝐴+𝐵𝐾)𝑆(𝐴+𝐵𝐾)+(𝐶+𝐷𝐾)𝑆(𝐶+𝐷𝐾)<0.(3.17) By Lemma 3.1, system (2.1) is mean square stabilizable.

4. Solutions to SARE and SDP

The following theorem will state the existence of the solution of the SARE (2.9) via SDP (3.6).

Theorem 4.1. The optimal set of (3.6) is nonempty, and any optimal solution 𝑃 must satisfy the SARE (2.9).

Proof. Since system (2.1) is mean square stabilizable, by Theorem 3.4, (3.7) is strictly feasible. Equation (3.6) is strictly feasible because 𝑃0 is a interior point of 𝒫. By Proposition 2.12, (3.6) is nonempty and 𝑃 satisfies 𝐴(𝑃)𝑍=0; that is, 𝑃+𝐴𝑃𝐴+𝐶𝑃𝐶+𝑄𝐴𝑃𝐵+𝐶𝑃𝐷𝐵𝑃𝐴+𝐷𝑃𝐶𝐵𝑃𝐵+𝐷𝑃𝐷+𝑅00𝑃𝑃0𝑆𝑈𝑈𝑇00𝑊=0.(4.1) From the above equality, we have the following equalities: 𝑃+𝐴𝑃𝐴+𝐶𝑃𝐴𝐶+𝑄𝑆+𝑃𝐵+𝐶𝑃𝐷𝑈=0,(4.2)𝑃+𝐴𝑃𝐴+𝐶𝑃𝑈𝐶+𝑄+𝐴𝑃𝐵+𝐶𝑃𝐷𝐵𝑇=0,(4.3)𝑃𝐴+𝐷𝑃𝐶𝐵𝑆+𝑃𝐵+𝐷𝑃𝐵𝐷+𝑅𝑈=0,(4.4)𝑃𝐴+𝐷𝑃𝐶𝑈+𝐵𝑃𝐵+𝐷𝑃𝑃𝐷+𝑅𝑇=0,(4.5)𝑃0𝑊=0.(4.6) Moreover, 𝑅+𝐷𝑃𝐷+𝐵𝑃𝐵>0 because of 𝑅+𝐷𝑃0𝐷+𝐵𝑃0𝐵>0 and 𝑃𝑃0. Then by (4.4), 𝑈=(𝐵𝑃𝐵+𝐷𝑃𝐷+𝑅)1(𝐵𝑃𝐴+𝐷𝑃𝐶)𝑆. Substituting it into (4.2) yields (𝑃)𝑆=0. Remember that 𝑆, 𝑇, 𝑈, 𝑊 satisfy the equality constraint in (3.7). Multiplying both sides by (𝑃), we have 𝑃𝐴𝑆𝐴+𝐶𝑆𝐶+𝐵𝑈𝐴+𝐷𝑈𝐶+𝐴𝑈𝐵+𝐶𝑈𝐷+𝐵𝑇𝐵+𝐷𝑇𝐷𝑃+𝑊+𝐼=0.(4.7) Considering 𝑆𝑈𝑈𝑇0, it follows from Lemma 2.8 that 𝑇𝑈𝑆+𝑈 and 𝑈=𝑈𝑆𝑆+. By Lemma 2.9, 𝑆+0 and 𝑆𝑆+=𝑆+𝑆. So we have 𝑃𝐴𝑆𝐴+𝐶𝑆𝐶+𝐵𝑈𝐴+𝐷𝑈𝐶+𝐴𝑈𝐵+𝐶𝑈𝐷+𝐵𝑈𝑆+𝑈𝐵+𝐷𝑈𝑆+𝑈𝐷𝑃𝑃=(𝐶𝑆+𝐷𝑈)𝑆+𝑆𝐶+𝑈𝐷+(𝐴𝑆+𝐵𝑈)𝑆+𝑆𝐴+𝑈𝐵𝑃0.(4.8) Then it follows that (𝑃)(𝑃)0. It yields (𝑃)=0 due to (𝑃)0.

The following theorem shows that any optimal solution of the primal SDP results in a stabilizing control for LQ problem.

Theorem 4.2. Let 𝑃 be an optimal solution to the SDP (3.6). Then the feedback control 𝑢(𝑡)=(𝑅+𝐵𝑃𝐵+𝐷𝑃𝐷)1(𝐵𝑃𝐴+𝐷𝑃𝐶)𝑥(𝑡) is mean square stabilizing for system (2.1).

Proof. Optimal dual variables 𝑆, 𝑇, 𝑈, 𝑊 satisfy (4.2)–(4.6). 𝑈=(𝐵𝑃𝐵+𝐷𝑃𝐷+𝑅)1(𝐵𝑃𝐴+𝐷𝑃𝐶)𝑆. Now we show 𝑆>0. Let 𝑆𝑥=0, 𝑥𝑛. The constraints in (3.7) imply 𝑥𝐴𝑆𝐴+𝐶𝑆𝐶+𝐵𝑈𝐴+𝐷𝑈𝐶+𝐴𝑈𝐵+𝐶𝑈𝐷+𝐵𝑈𝑆+𝑈𝐵+𝐷𝑈𝑆+𝑈𝐷+𝑊+𝐼𝑥0.(4.9) Similar to the proof of Theorem 4.1, we have 𝑥=0. We conclude that 𝑆>0 from 𝑆0. Again by the equality constraint in (3.7), we have 𝑆+𝐴𝑆𝐴+𝐶𝑆𝐶+𝐵𝑈𝐴+𝐷𝑈𝐶+𝐴𝑈𝐵+𝐶𝑈𝐷+𝐵𝑈𝑆1𝑈𝐵+𝐷𝑈𝑆1𝑈𝐷=𝑆+𝐴+𝐵𝑈𝑆1𝑆𝐴+𝐵𝑈𝑆1+𝐶+𝐷𝑈𝑆1𝑆𝐶+𝐷𝑈𝑆1<0.(4.10) By Lemma 3.1, the above inequality is equivalent to the mean square stabilizability of system (2.1) with 𝑢(𝑡)=𝑈𝑆1𝑥(𝑡)=(𝑅+𝐵𝑃𝐵+𝐷𝑃𝐷)1(𝐵𝑃𝐴+𝐷𝑃𝐶)𝑥(𝑡). This ends the proof.

Theorem 4.3. There is a unique optimal solution to (3.6), which is the maximal solution to SARE (2.9).

Proof. The proof is similar to Theorem  9 in [11] and is omitted.

Theorem 4.4. Assume that 𝒫 is nonempty, then SARE (2.9) has a maximal solution 𝑃max, which is the unique optimal solution to SDP: maxTr(𝑃)subjectto𝐴(𝑃)=𝑃+𝐴𝑃𝐴+𝐶𝑃𝐶+𝑄𝐴𝑃𝐵+𝐶𝐵𝑃𝐷𝑃𝐴+𝐷𝑃𝐶𝑅+𝐵𝑃𝐵+𝐷𝑃𝐷𝑅+𝐵𝑃𝐵+𝐷𝑃D>0.0,(4.11)

Proof. The first assertion is Theorem  4 in [16]. As to the second part, we proceed as follows. By Lemma 2.7, 𝑃max satisfies the constraints in (4.11). 𝑃max is an optimal solution to (4.11) due to the maximality. Next we prove the uniqueness. Assume that 𝑃max is another optimal solution to (4.11). Then we have Tr(𝑃max)=Tr(𝑃max). According to Definition 2.4, 𝑃max𝑃max. This yields 𝑃max=𝑃max. Hence, the proof of the theorem is completed.

Remark 4.5. Here we drop the assumption that the interior of 𝒫 is nonempty.

Remark 4.6. Theorem 4.4 presents that the maximal solution to SARE (2.9) can be obtained by solving SDP (4.11). The result provides us a computational approach to the SARE. Furthermore, as shown in [16], the relationship between the LQ value function and the maximal solution to SARE (2.9) can be established; that is, assuming that 𝒫 is nonempty, then the value function 𝑉(𝑥0)=𝑥0𝑃max𝑥0 and the optimal control can be expressed as 𝑢(𝑡)=(𝑅+𝐵𝑃max𝐵+𝐷𝑃max𝐷)1(𝐵𝑃max𝐴+𝐷𝑃max𝐶)𝑥(𝑡).
The above results represent SARE (2.9) may exist a solution even if 𝑅 is indefinite (even negative definite). To describe the allowable negative degree, we give the following definition to solvability margin.

Definition 4.7 (see [11]). The solvability margin 𝑟 is defined as the largest nonnegative scalar 𝑟0 such that (2.9) has a solution for any 𝑅>𝑟𝐼.

By Theorem 4.4, the following conclusion is obvious.

Theorem 4.8. The solvability margin 𝑟 can be obtained by solving the following SDP: max𝑟subjectto𝑃+𝐴𝑃𝐴+𝐶𝑃𝐶+𝑄𝐴𝑃𝐵+𝐶𝐵𝑃𝐷𝑃𝐴+𝐷𝑃𝐶𝐵𝑃𝐵+𝐷𝐵𝑃𝐷𝑟𝐼0,𝑃𝐵+𝐷𝑃𝐷𝑟𝐼>0,𝑟>0.(4.12)

5. Numerical Examples

Consider the system (2.1) with 𝐴, 𝐵, 𝐶, 𝐷 as follows:,.𝐴=0.43262.28771.18921.66561.14650.03760.12531.19090.3273,𝐵=0.81470.91340.90580.63240.12700.0975𝐶=0.17460.58830.11390.18672.18321.06680.72580.13640.0593,𝐷=0.27850.96490.54690.15760.95750.9706(5.1)

5.1. Mean Square Stabilizability

In order to test the mean square stabilizability of system (2.1), we only need to check weather or not LMI (3.5) is feasible by Theorem 3.2. Making use of LMI feasp solver [22], we find matrices 𝑃 and 𝑈 satisfying (3.5):𝑃=4.82030.21052.07570.21051.28122.22872.07572.22875.3885,𝑈=5.41850.92281.31134.54990.94260.5770,(5.2) and the stabilizing control 𝑢(𝑡)=𝑈𝑃1𝑥(𝑡); that is,𝑢(𝑡)=0.66062.11160.86230.96790.40790.0970𝑥(𝑡).(5.3)

5.2. Solutions of SARE

Let 𝑄=𝐼 in SARE (2.9). Below, we solve SARE (2.9) via the SDP (4.11) (LMI mincx solver [22]).

Case 1. 𝑅 is positive definite. Choose 𝑅=2112, and we obtain the maximal solution to (2.9): 𝑃max=44.2780100.574448.8271100.5744360.6942182.457248.8271182.457294.6103.(5.4)

Case 2. 𝑅 is indefinite. Choose 𝑅=0.20.30.30.2, and we obtain the maximal solution to (2.9): 𝑃max=17.797540.097319.190040.0973135.218967.679019.190067.679035.5117.(5.5)

Case 3. 𝑅 is negative definite. First we can get the solvability margin 𝑟=0.4936 by solving SDP (4.12) (LMI gevp solver [22]). Hence (2.9) has a maximal solution when 𝑅>0.4936𝐼. Choose 𝑅=0.4𝐼, and we obtain the maximal solution to (2.9): 𝑃max=29.628366.789732.243166.7897234.0792117.906532.2431117.906561.3301.(5.6)

6. Conclusion

In this paper, we use the SDP approach to the study of discrete-time indefinite stochastic LQ control. It was shown that the mean square stabilization of system (2.1) is equivalent to the strict feasibility of the SDP (3.7). In addition, the relation between the optimal solution of (3.6) and the maximal solution of SARE (2.9) has been established. What we have obtained can be viewed as a discrete-time version of [11]. Of course, there are many open problems to be solved. For example, 𝑅+𝐵𝑃𝐵+𝐷𝑃𝐷>0 is a basic assumption in this paper. A natural question is whether or not we can weaken it to 𝑅+𝐵𝑃𝐵+𝐷𝑃𝐷0. This problem merits further study.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (61174078), Specialized Research Fund for the Doctoral Program of Higher Education (20103718110006), and Key Project of Natural Science Foundation of Shandong Province (ZR2009GZ001).

References

  1. W. M. Wonham, “On a matrix Riccati equation of stochastic control,” SIAM Journal on Control and Optimization, vol. 6, no. 4, pp. 681–697, 1968.
  2. C. E. De Souza and M. D. Fragoso, “On the existence of maximal solution for generalized algebraic Riccati equations arising in stochastic control,” Systems & Control Letters, vol. 14, no. 3, pp. 233–239, 1990. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  3. P. J. McLane, “Optimal stochastic control of linear systems with state- and control-dependent disturbances,” IEEE Transactions on Automatic Control, vol. 16, no. 6, pp. 793–798, 1971. View at Publisher · View at Google Scholar
  4. W. H. Zhang and B. S. Chen, “On stabilizability and exact observability of stochastic systems with their applications,” Automatica, vol. 40, no. 1, pp. 87–94, 2004. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  5. W. H. Zhang, H. S. Zhang, and B. S. Chen, “Generalized Lyapunov equation approach to state-dependent stochastic stabilization/detectability criterion,” IEEE Transactions on Automatic Control, vol. 53, no. 7, pp. 1630–1642, 2008. View at Publisher · View at Google Scholar
  6. F. Carravetta and G. Mavelli, “Suboptimal stochastic linear feedback control of linear systems with state- and control-dependent noise: the incomplete information case,” Automatica, vol. 43, no. 5, pp. 751–757, 2007. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  7. W. Fleming and R. Rishel, Deterministic and Stochastic Optimal Control, Springer, Berlin, Germany, 1975.
  8. M. H. A. Davis, Linear Estimation and Stochastic Control, Chapman and Hall, London, UK, 1977, Chapman and Hall Mathematics Serie.
  9. S. P. Chen, X. J. Li, and X. Y. Zhou, “Stochastic linear quadratic regulators with indefinite control weight costs,” SIAM Journal on Control and Optimization, vol. 36, no. 5, pp. 1685–1702, 1998. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  10. M. A. Rami, X. Chen, J. B. Moore, and X. Y. Zhou, “Solvability and asymptotic behavior of generalized Riccati equations arising in indefinite stochastic LQ controls,” IEEE Transactions on Automatic Control, vol. 46, no. 3, pp. 428–440, 2001. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  11. M. A. Rami and X. Y. Zhou, “Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls,” IEEE Transactions on Automatic Control, vol. 45, no. 6, pp. 1131–1143, 2000. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  12. M. A. Rami, J. B. Moore, and X. Y. Zhou, “Indefinite stochastic linear quadratic control and generalized differential Riccati equation,” SIAM Journal on Control and Optimization, vol. 40, no. 4, pp. 1296–1311, 2001. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  13. X. Li, X. Y. Zhou, and M. A. Rami, “Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon,” Journal of Global Optimization, vol. 27, no. 2-3, pp. 149–175, 2003. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  14. X. Li and X. Y. Zhou, “Indefinite stochastic LQ controls with Markovian jumps in a finite time horizon,” Communications in Information and Systems, vol. 2, no. 3, pp. 265–282, 2002. View at Zentralblatt MATH
  15. M. A. Rami, X. Chen, and X. Y. Zhou, “Discrete-time indefinite LQ control with state and control dependent noises,” Journal of Global Optimization, vol. 23, no. 3-4, pp. 245–265, 2002. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  16. J. J. Qi and W. H. Zhang, “Discrete-time indefinite stochastic LQ optimal control: infinite horizon case,” Acta Automatica Sinica, vol. 35, no. 5, pp. 613–617, 2009.
  17. Y. L. Huang, W. H. Zhang, and H. S. Zhang, “Infinite horizon linear quadratic optimal control for discrete-time stochastic systems,” Asian Journal of Control, vol. 10, no. 5, pp. 608–615, 2008. View at Publisher · View at Google Scholar
  18. D. D. Yao, S. Zhang, and X. Y. Zhou, “Stochastic linear-quadratic control via semidefinite programming,” SIAM Journal on Control and Optimization, vol. 40, no. 3, pp. 801–823, 2001. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  19. V. Balakrishnan and L. Vandenberghe, “Semidefinite programming duality and linear time-invariant systems,” IEEE Transactions on Automatic Control, vol. 48, no. 1, pp. 30–41, 2003. View at Publisher · View at Google Scholar
  20. L. Vandenberghe and S. Boyd, “Semidefinite programming,” SIAM Review, vol. 38, no. 1, pp. 49–95, 1996. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  21. A. El Bouhtouri, D. Hinrichsen, and A. J. Pritchard, “H-type control for discrete-time stochastic systems,” International Journal of Robust and Nonlinear Control, vol. 9, no. 13, pp. 923–948, 1999. View at Zentralblatt MATH
  22. S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory, SIAM, Philadelphia, Pa, USA, 1994.