Abstract

This paper studies the discrete-time stochastic linear quadratic (LQ) problem with a second moment constraint on the terminal state, where the weighting matrices in the cost functional are allowed to be indefinite. By means of the matrix Lagrange theorem, a new class of generalized difference Riccati equations (GDREs) is introduced. It is shown that the well-posedness, and the attainability of the LQ problem and the solvability of the GDREs are equivalent to each other.

1. Introduction

LQ control, initiated by Kalman [1] and extended to stochastic systems by Wonham [2], is one of the most important classes of optimal control issues from both theory and application point of view; we refer the reader to [28] for representative work in this area. Different from the classical LQ in modern control theory, it was found in [9, 10] that a stochastic LQ problem with indefinite control weighting matrices can still be well-posed, which evoked a series of subsequent researches; see, for example, [11, 12].

It is well known that in practical engineering, the system state and control input are always subject to various constraints, so how to solve the constrained stochastic LQ issue is a more attractive topic; we refer the reader to [1319]. Reference [14] presented a tractable approach for LQ controller design of the system with additive noise. Reference [16] was about the constrained LQ of deterministic systems with state equality constraints. Reference [13] studied the parametrization of the solutions of finite-horizon constrained LQ control. Reference [15] was devoted to a stochastic LQ optimal control and an application to portfolio selection, where the control variable is confined to a cone, and all the coefficients of the state equation are random processes. Reference [19] studied the indefinite stochastic LQ control problem of continuous-time Itô systems with a linear equality constraint on the terminal state and gave a necessary condition for the existence of an optimal controller. Reference [20] generalized the results of [19] to discrete-time stochastic systems.

In this paper, different from [19, 20] on the constraint conditions, we would like to deal with stochastic LQ control of discrete-time multiplicative noise systems with a second moment constraint and such constraints are often encountered in filtering design; see [21, 22]. By means of Lagrange theorem, we present a necessary condition for the existence of an optimal linear state feedback control with the second moment constraint on the terminal states. It is proved that the solvability of GDRE is necessary and sufficient for the existence of an optimal control under either of the state feedback case or of the open-loop forms. Moreover, we show that the well-posedness and the attainability of the constrained LQ problem, the feasibility of the LMI, and the solvability of the GDRE are equivalent to each other. The novel contribution of this paper is to consider a constrained discrete-time LQ optimal stochastic control, which includes some results of [23] as special cases. A new class of generalized difference Riccati equations (GDREs) is first introduced.

The remainder of the paper is organized as follows. Section 2 gives some definitions and preliminaries. In Section 3, the optimal state feedback control is studied using the matrix Lagrange theorem. We give a necessary and sufficient condition for the well-posedness of the constrained LQ control in Section 4. Section 5 shows the equivalence among the well-posedness and the attainability of the LQ problem, the feasibility of the LMI, and the solvability of the GDRE. The set of all optimal controls is determined. We conclude the paper in Section 6.

Throughout the paper, the following notations are adopted: denotes the transpose of . : is a positive definite (positive semidefinite) symmetric matrix. : the trace of a square matrix . : the space of all matrices. : the space of all symmetric matrices.

2. Problem Setting

Consider the following constrained discrete-time stochastic LQ control problem.

Problem 1. Consider where the state , the control input , and the noise , , The process is a sequence of second-order stationary random variables defined on a complete probability space . Without loss of generality, we assume that where is the Kronecker delta, . is a constant, , and are matrices having appropriate dimensions determined from context, and and are real symmetric indefinite matrices. is a given deterministic vector.

Definition 2. Problem 1 is called well-posed, if ,

Definition 3. Problem 1 is called attainable, if , there exists a sequence , such that . In this case, is called an optimal control sequence.
Now, let us consider a mathematical programming (MP) problem in a matrix space:

Definition 4. Let be a point satisfying and then is said to be a constraint regular point if the gradient vectors , are linearly independent.

Lemma 5 (Lagrange theorem [24]). Assume that the functions , , are twice continuously differentiable. If a regular point is also a relative minimum point for the original MP, then there exists a vector such that where the Lagrangian function .

3. A Necessary Condition for State Feedback Control

In this section, by the matrix Lagrange theorem, we present a necessary condition for Problem 1 based on a new type of GDREs.

Let . Through a simple calculation, the following deterministic optimal control Problem 6 is equivalent to the original Problem 1 under the state feedback for .

Problem 6. Consider with

Remark 7. If Problem 1 has a linear feedback optimal control solution , , then , , are the optimal solution of Problem 6.

Theorem 8. If Problem 1 is attainable by , and the regular point is a locally optimal solution of Problem 6, then there exist symmetric matrices , and solving the following GDRE: where is the Moore-Penrose generalized inverse of . Moreover, with , , being any given real matrices:

To prove Theorem 8, we mainly use Lemma 5 to Problem 6 together with the following lemma to obtain GDRE (10) and then apply the technique of completing squares to show (12).

Lemma 9 (see [12]). Let be given matrices with appropriate sizes; then the matrix equation has a solution if and only if Moreover, any solution to can be represented by where is any matrix with appropriate size.

Proof. According to Remark 7, is also the optimal solution of Problem 6. Problem 6 is a typical MP problem about and as follows: where Let matrices , , be the Lagrangian multipliers of and let be the Lagrangian multiplier of ; then the Lagrangian function According to the the matrix Lagrange theorem, we obtain Based on the partial rule of gradient matrices, (20) can be transformed into Let Then we obtain Applying Lemma 9, we have Equation (21) yields Substituting into (26), it follows Without loss of generality, we can assume that is symmetric. Otherwise, we can take . The objective functional A completion of square implies We assert that must satisfy If it is not so, there is for with a negative eigenvalue . Denote the unitary eigenvector with respect to by . Let be an arbitrary scalar; we construct a control sequence as follows: The associated cost functional becomes Let ; then , which contradicts the attainability of Problem 1. So (30) holds.
In view of (29) and (30), (11) and (12) are easily derived. The proof is completed.

Remark 10. In Theorem 8, in order to apply matrix Lagrange theorem, we assume the optimal solution is a regular point. Generally speaking, for a given LQ control, it is easy to examine the regular condition.
Below, we present a numerical example to illustrate the effectiveness of Theorem 8.

Example 11. In Problem 1, we set The state and control weighting matrices are as By the relationship between Problems 1 and 6, we know Applying Theorem 8, we obtain Stage 2. Consider Stage 1. Consider The optimal cost value of Problem 1 is We are able to test the regular condition of as follows. In Problem 6, which is linear about and quadratic about , while is linear about . By simple calculations, , , and are all nonzero vectors and hence are linearly independent.

4. Well-Posedness

In this section, we first establish the link between the well-posedness of Problem 1 and the feasibility of some LMIs and then prove that the solvability of GDRE (10) is not only necessary but also sufficient to the well-posedness of Problem 1. Moreover, the well-posedness and the attainability of Problem 1, the feasibility of some LMIs, and the solvability of GDRE (10) are equivalent to each other.

Theorem 12. Problem 1 is well-posed if there exist symmetric matrices , and solving the following LMIs: where

Proof. Note that By (41), it is easy to deduce that the cost functional is bounded from below by Hence, Problem 1 is well-posed.

Remark 13. Theorem 12 tells us that any symmetric matrices , and satisfying LMIs (41)-(42) provide a lower bound for the cost function. In what follows, we will show that this lower bound is an exact optimal cost value if and solve GDRE (10).
We have shown that if the LMIs (41)-(42) are satisfied, then the constrained LQ Problem 1 is well-posed. Below, we further show some other equivalent conditions.

Lemma 14 (extended Schur’s lemma [25]). Let the matrices , , be given with appropriate sizes. Then, the following three conditions are equivalent:

Theorem 15. Problem 1 is well-posed if and only if there exist symmetric matrices , and satisfying GDRE (10). Furthermore, the optimal cost is
A key to prove Theorem 15 is the necessity part, where the stochastic optimization principle is used.

Proof. Necessity. For , define By the stochastic optimization principle, when is finite, then so is for any . Since Problem 1 is assumed to be well-posed at , is finite at any stage . Now let us start with , and let , and we have Since is finite, using Lemma 4.3 of [23], there exists a symmetric matrix such that The obtained solution sequence of symmetric matrices , , and to GDRE (10) satisfy Then by the stochastic optimality principle, the following holds: Lemma 4.3 of [23] provides necessary and sufficient conditions for the finiteness of : Moreover, The above proves the necessity part via mathematical induction.
Sufficiency. From the proof of Theorem 8, if GDRE (10) admits a solution and , Problem 1 is not only well-posed, but also attainable. The proof of this theorem is complete.

5. Other Equivalent Conditions

In this section, we present some other equivalent conditions for Problem 1.

Theorem 16. For the constrained LQ Problem 1, the following are equivalent:(i)Problem 1 is well-posed.(ii)Problem 1 is attainable.(iii)The LMIs (41)-(42) are feasible.(iv)The GDRE (10) is solvable.
Furthermore, when any one of the above conditions is satisfied, Problem 1 is attainable by where , are solutions to the GDRE (10).

Proof. Applying Theorems 1215, . is shown by Theorem 8. The rest is to prove and (56). Let , solve the GDRE (10). In view of a completion of squares yields which shows
Finally, we present a general expression for the optimal control set based on the solution to GDRE (10).

Theorem 17. Assume that the GDRE (10) admits a solution. Then the set of all optimal controls is determined by where and are arbitrary random variables defined on the probability space . Moreover, the optimal cost value is uniquely given by where , and are the solution to the GDRE (10).

Proof. This theorem can be proved by repeating the same procedure as in Theorem 5.1 of [23].

6. Conclusion

In this paper, we have investigated a class of indefinite stochastic LQ control problems with second moment constraints on the terminal state. By the matrix Lagrange theorem, we have established a new GDRE (10) associated with the constrained optimization Problem 1. In addition, by introducing LMIs (41)-(42), we show that the well-posedness and the attainability of Problem 1, the feasibility of the LMIs (41)-(42), and the solvability of GDRE (10) are equivalent to each other.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This work is supported by the National Natural Science Foundation of China (no. 61174078), the Research Fund for the Taishan Scholar Project of Shandong Province of China and SDUST Research Fund (no. 2011KYTD105), and State Key Laboratory of Alternate Electrical Power System with Renewable Energy Sources (Grant no. LAPS13018).