Indefinite LQ Optimal Control with Terminal State Constraint for Discrete-Time Uncertain Systems

Abstract

Uncertainty theory is a branch of mathematics for modeling human uncertainty based on the normality, duality, subadditivity, and product axioms. This paper studies a discrete-time LQ optimal control with terminal state constraint, whereas the weighting matrices in the cost function are indefinite and the system states are disturbed by uncertain noises. We first transform the uncertain LQ problem into an equivalent deterministic LQ problem. Then, the main result given in this paper is the necessary condition for the constrained indefinite LQ optimal control problem by means of the Lagrangian multiplier method. Moreover, in order to guarantee the well-posedness of the indefinite LQ problem and the existence of an optimal control, a sufficient condition is presented in the paper. Finally, a numerical example is presented at the end of the paper.

1. Introduction

The linear quadratic (LQ) optimal control problem has been pioneered by Kalman [1] for deterministic systems, which is extended to stochastic systems by Wonham [2], and has rapid development in both theory and application [3]. Usually, it is an assumption that the control weighting matrix in the cost is strictly definite. For stochastic LQ optimal control, it is first revealed in [4] that even if the state and control weighting matrices are indefinite the corresponding problem may be still well-posed, which evoked a series of subsequent researches in continuous time [5] and in discrete-time [6]. In fact, some constraints are of considerable importance in many physical systems; the system state and control input are always subject to various constraints, so the constrained stochastic LQ issue has a concrete application background. For that reason, some researchers discussed stochastic LQ optimal problems with indefinite control weights and constraints [7, 8].

As is well known, these stochastic optimal control problems have been well studied by probability theory which is based on a large number of sample sizes. Sometimes, no samples are available to estimate the probability distribution. For such situation, we have to invite some domain experts to evaluate the belief degree that each event will occur. In order to rationally deal with belief degrees, uncertainty theory was established by Liu [9] in 2007 and refined by Liu [10] in 2010. Nowadays, uncertainty theory has become a new branch of mathematics for modeling indeterminate phenomena, which has been well developed and applied in a wide variety of real problems: option pricing problem [11], facility location problem [12], inventory problem [13], assignment problem [14], and production control problem [15].

Based on the uncertainty theory, Zhu [16] proposed an uncertain optimal control model in 2010 and gave an equation of optimality as a counterpart of Hamilton-Jacobi-Bellman equation. After that, some uncertain optimal control problems have been solved. As such, Sheng and Zhu [17] investigated an optimistic value model of uncertain optimal control problem; Yan and Zhu [18] established an uncertain optimal control model for switched systems. Inspired by the preceding work, we will tackle an indefinite LQ optimal control with terminal state constraint for discrete-time uncertain systems, which is a constrained uncertain optimal control problem. The rest of the paper is organized as follows. Section 2 collects some preliminary results. In Section 3, an indefinite LQ optimal control with terminal state constraint is discussed. We present a general expression for the optimal control set in Section 4. A numerical example is applied in Section 5 to demonstrate the effectiveness of the model. We conclude the paper in Section 6.

For convenience, throughout the paper, we adopt the following notations: is the real -dimensional Euclidean space; is the set of all matrices; is the transpose of matrix ; and is the trace of a square matrix . Moreover, (resp., ) means that and is positive (resp., positive semidefinite) definite.

2. Some Preliminaries

In this section, we introduce some useful definitions about uncertainty theory and Moore-Penrose pseudoinverse of a matrix.

Let be a nonempty set, and let be a -algebra over . Each element in is called an event. An uncertain measure was defined by Liu [9] via the following three axioms.

Axiom 1 (normality axiom). for the universal set .

Axiom 2 (duality axiom). for any event .

Axiom 3 (subadditivity axiom). For every countable sequence of events , we have

The triplet is called an uncertainty space. Furthermore, Liu [19] defined a product uncertain measure by the product axiom.

Axiom 4 (product axiom). Let be uncertainty spaces for . Then, the product uncertain measure on the product -algebra satisfieswhere are arbitrarily chosen events from for , respectively.

An uncertain variable is defined by Liu [9] as a function from an uncertainty space to the set of real numbers such that is an event for any Borel set . In addition, an uncertainty distribution of is defined asfor any real number .

Independence is an important concept in uncertainty theory. The uncertain variables are said to be independent (Liu [19]) iffor any Borel sets of real numbers.

An uncertain variable is called linear (Liu [9]) if it has a linear uncertainty distribution denoted by , where and are real numbers with .

Let be an uncertain variable. Then, the expected value (Liu [9]) of is defined by provided that at least one of the two integrals is finite.

Remark 1. For numbers and , if and are independent uncertain variables. Generally speaking, the expected value operator is not necessarily linear if the independence is not assumed.

Remark 2. Let where are uncertain variables for ,  . The expected value of is provided by

Lemma 3 (Penrose [20]). Let a matrix be given. Then, there exists a unique matrix such that The matrix is called the Moore-Penrose pseudoinverse of .

Lemma 4 (Penrose [20]). Let matrices L, M, and N be given with appropriate sizes. Then, the matrix equation has a solution X if and only if Moreover, any solution to is represented by , where is a matrix with an appropriate size.

3. Indefinite LQ Optimal Control with Constraints

3.1. Problem Statement

Consider the following indefinite LQ optimal control with terminal state constraint for discrete-time uncertain systems: where , state , control input , , and is a given crisp vector. Denote . Moreover, and are real symmetric matrices with appropriate dimensions. In addition, is a constant; the coefficients and are crisp matrices having appropriate dimensions determined from context. Besides, the noises are independent linear uncertain variables with the distribution

In this paper, the weighting matrices in the objective functional are not required to be definite. Therefore, problem (10) is an indefinite LQ optimal control problem. Next, we give the following definitions.

Definition 5. The indefinite LQ problem (10) is called well-posed if

Definition 6. A well-posed problem is called solvable, if, for , there is a control sequence () that achieves . In this case, the control () is called an optimal control sequence.

3.2. An Equivalent Problem

Next, we transform the uncertain LQ optimal control problem (10) into an equivalent deterministic LQ optimal control problem which is subject to a matrix difference equation constraint.

Let . Since state , is matrix whose elements are uncertain variables, and is a symmetric crisp matrix . Denote , where are matrices for .

Theorem 7. If the indefinite LQ problem (10) is solvable by a feedback control where are constant crisp matrices, then it is equivalent to the following deterministic optimal control problem:for .

Proof. Assume that the indefinite LQ problem (10) is solvable by a feedback control for . Let for . Then, we havewhere Then, we obtain that . Because and are not independent, we know that We will deal with (18) as follows.(i)If , we obtain(ii)If , we know that and . According to Example  2 in [21], we have Therefore, we have Substituting (21) into (16) produces the following state matrix:The associated cost function reduces toand the constraint becomes

Remark 8. Obviously, if problem (10) has a linear feedback optimal control solution , then is the optimal solution of problem (14).

3.3. A Necessary Condition for State Feedback Control

In this subsection, a necessary condition for the optimal linear state feedback control with deterministic gains to the indefinite LQ problem (10) is obtained by applying the deterministic matrix maximum principle [22].

Theorem 9. If the indefinite LQ problem (10) is solvable by a feedback controlwhere are constant crisp matrices, then there exist symmetric matrices and a nonnegative solving the following constrained difference equation:for . Moreover,with , , being any given crisp matrices.

Proof. Assume that the indefinite LQ problem (10) is solvable by where the matrices are viewed as the control to be determined. It is obvious that is also the optimal solution of problem (14) which is deterministic LQ optimal control problem. Hence, we can apply the matrix Lagrangian multiplier method to solve problem (14).
Let matrices be the Lagrange multipliers of , and let be the Lagrange multiplier of . Then, the Lagrange function is formed aswhere According to the first-order necessary conditions for optimality [22], we haveBased on the partial rule of gradient matrices [22], (30) can be transformed into Let Then, (33) can be rewritten as . Applying Lemma 4, we have , and For (31), according to we have Substituting (35) into (37), we obtain Consider the objective functional Since , the objective functional can be rewritten as By applying (32) and Lemma 3, a completion of square impliesWe assert that () must satisfy If it is not so, there is an for with a negative eigenvalue . Denote the unitary eigenvector with respect to as (i.e., and ). Let be an arbitrary scalar and construct a control sequence as follows:The associated cost functional becomes Let . Then, , which contradicts the well-posedness of problem (10).

3.4. Special Cases

We have obtained that in the constrained difference equation (25) of Theorem 9. The following corollaries are special cases of the above result if we have and .

Corollary 10. The indefinite LQ problem (10) is uniquely solvable if and only if for . Moreover, the unique optimal control is given by

Proof. By using Theorem 9, we immediately obtain the corollary.

Corollary 11. If for , then any admissible control of the indefinite LQ problem (10) is optimal and the constrained difference equation (25) reduces to the following linear system:for

Proof. Letting in (25), it is easy to obtain the linear system (46). Letting in (41), (41) is simplified aswhich implies that for any admissible control. Then, any admissible control of the indefinite LQ problem (10) is optimal.

3.5. Well-Posedness of the Indefinite LQ Problem

In the following, it is shown that the solvability of the constrained difference equation (25) is sufficient for the well-posedness of the indefinite LQ problem and the existence of an optimal control. Moreover, any optimal control can be represented explicitly as a linear state feedback by the solution of (25).

Theorem 12. The indefinite LQ problem (10) is well-posed if there exist symmetric matrices and satisfying the constrained difference equation (25). Moreover, the optimal control is given byFurthermore, the optimal cost of the indefinite LQ problem (10) is

Proof. Let and satisfy (25). Then,By applying Lemma 3, a completion of square impliesSince , from (51), we can easily deduce that the cost function of problem (10) is bounded from below by Hence, the indefinite LQ problem (10) is well-posed. It is clear that it is solvable by the feedback controlFurthermore, by using and which we have obtained in Theorems 7 and 9, (52) indicates that the optimal value of problem (10) equals

4. General Expression for the Optimal Control Set

In this part, we will present a general expression for the optimal control set based on the solution to (25).

Theorem 13. Assume that and solves the constrained difference equation (25). A sufficient and necessary condition that is in the set of all optimal feedback controls for indefinite LQ problem (10) is thatwhere and are arbitrary variables with appropriate size.

Proof.
Sufficiency. According to the same calculation as in Theorem 9, we haveBy denoting and , we obtainAccording to (56) and (57), we obtain As , we know that the control minimizes with the optimal value for .
Necessity. If any control sequence which minimizes the cost function , then we havefor The above equality implies that Since , we get the following equivalent condition: We see that solves the following equation:By using Lemma 3 with , , , it is easy to verify that Then, we obtain the solution of  (62) with As in (35), the optimal control can be represented by

5. Numerical Example

In this section, application of Theorem 9 to solve constraint optimal control problem is illustrated. We present a two-dimensional indefinite LQ problem with terminal state constraint for discrete-time uncertain systems. A set of specific parameters of the coefficients are given as follows: The state weights and the control weights are as follows:Note that, in this example, the state weight is negative semidefinite, is negative definite, and is positive semidefinite and the control weights and are negative definite.

In order to find the optimal controls and optimal cost value of this example, we have to solve the following equations:

Firstly, we haveThen, we get by solving (68), and we obtain

Secondly, by applying Theorem 9, we obtain the optimal feedback control and optimal cost value as follows.

For , we obtainThe optimal feedback control is , where

For , we obtainThe optimal feedback control is , where

Finally, the optimal cost value is

6. Conclusion

We have considered the indefinite LQ optimal control with terminal state constraint involving state and control dependent uncertain noises. We first transform the uncertain LQ optimal control problem into a deterministic LQ optimal control problem. By means of the matrix maximum principle, we have presented a necessary condition for the existence of optimal linear state feedback control. Besides, we have proved the well-posedness of the indefinite LQ constraint problem by applying the technique of completing squares. For further work, we will consider discrete-time indefinite LQ optimal control model with inequality constraint.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This work is supported by the National Natural Science Foundation of China (no. 61203050) and the Natural Science Research Project of the Education Bureau of Anhui Province (China) (no. KJ2015A076).