Abstract
We consider an optimal control problem subject to the terminal state equality constraint and continuous inequality constraints on the control and the state. By using the control parametrization method used in conjunction with a time scaling transform, the constrained optimal control problem is approximated by an optimal parameter selection problem with the terminal state equality constraint and continuous inequality constraints on the control and the state. On this basis, a simple exact penalty function method is used to transform the constrained optimal parameter selection problem into a sequence of approximate unconstrained optimal control problems. It is shown that, if the penalty parameter is sufficiently large, the locally optimal solutions of these approximate unconstrained optimal control problems converge to the solution of the original optimal control problem. Finally, numerical simulations on two examples demonstrate the effectiveness of the proposed method.
1. Introduction
Constrained optimal control problems often arise in a wide range of practical applications, including the swing minimization of transferring container [1], the flight maximization with a heating constraint [2], the fuel minimization of the soft landing of moon [3], and the fuel minimization of spacecraft rendezvous with collision avoidance constraint [4]. In these different applications, two types of constraints are often considered. The first one is terminal state equality constraint, which depends only on the final state of the system, whereas the other one is continuous inequality constraints on the control and the state, which restrict control and state at every point in the time horizon. Of these two types, continuous inequality constraints are by far the most difficult, as they include an infinite number of inequality constraints.
Most constrained optimal control problems are much too complex to obtain analytical solutions. Thus, they can only be solved by some numerical methods, such as the discretization method [5–7], the nonsmooth Newton method [8–10], and the control parametrization method [11]. Among these numerical methods, the control parametrization method is an efficient one for dealing with constrained optimal problems. Its main idea is that only the control variables are approximated as piecewise constant functions or linear functions while the system states remain unchanged. In [11], this kind of method used in conjunction with a time scaling transform is used to solve optimal control problems subject to continuous state inequality constraints, where the continuous state inequality constraints are handled by the constraint transcription method developed in [12]. It is extended in [13] to a more general case where both the state and control appear explicitly in the continuous inequality constraints. After the application of the control parameterization used in conjunction with the time scaling transform, a constrained optimal parameter selection problem is obtained. Now by using the constraint transcription method, the constrained optimal parameter selection problem can be approximated as a sequence of unconstrained optimal control problems, where two adjustable parameters are involved in these approximate problems—one controls the accuracy and the other one controls the feasibility. It is shown that, for any positive accuracy parameter, if the feasible parameter is sufficiently small, the obtained solution will satisfy the continuous inequality constraints. As the accuracy parameter approaches to zero, the global optimal solution of the approximate optimal control problem will converge to the global optimal solution of the original problem. However, since there are two parameters that need be adjusted, the convergence speed of the method may be slow. In addition, the approximate optimal control problem is usually nonconvex, and only a local optimal solution can be obtained, and it is not known if this local optimal solution will converge to a local optimal solution of the original problem.
In this paper, we develop a new computational approach based on the control parametrization method [13] and the exact penalty function method [14] for solving an optimal control problem subject to terminal state equality constraint and continuous inequality constraints on the state and the control. After the control parametrization, together with the time scaling transform, the constrained optimal control problem is approximated by an optimal parameter selection problem with the terminal state equality constraint and continuous inequality constraints on the control and state. A simple exact penalty function method is used to construct the constraint violation function from the terminal state equality constraint and continuous inequality constraints on the control and state, which is added to the object function to form a new one. Thus, the constrained optimal parameter selection problem is transformed into a sequence of unconstrained optimal control problems, where only one penalty parameter is involved in the approximation. It is shown that, if the penalty parameter value is sufficiently large, any local minimizer of the unconstrained optimal control problem is a local minimizer of the constrained optimal control problem. Two numerical examples are provided to illustrate the effectiveness of the proposed method.
The rest of the paper is organized as follows. In Section 2, based on the nonlinear dynamic model, a constrained optimal control problem is formulated. In Section 3, by using the control parametrization method used in conjunction with a time scaling transform, the constrained optimal control problem is approximated by a constrained optimal parameter selection problem. In Section 4, a simple penalty function method is applied to transform the constrained optimal parameter selection problem into a sequence of unconstrained optimal control problems. In Section 5, it is shown that if the penalty parameter is sufficiently large, the solutions of these approximate unconstrained optimal control problems locally converge to the solution of the original problem. A numerical algorithm is developed to obtain the solution of the original problem through solving a sequence of unconstrained optimal control problems. In Section 6, A pair of numerical simulations are provided to demonstrate the effectiveness of the proposed method. Finally, in Section 7, some concluding remarks are stated.
2. Problem Formulation
Consider the following nonlinear dynamical system: with the initial condition and the terminal condition respectively, where is the terminal time, and are, respectively, control and state vectors at time , is the given initial state, and is the given terminal state. is a continuously differentiable function with respect to all its arguments, and there exists a constant such that where and is the Euclidean norm.
Define where and are given real numbers. A piecewise-continuous function satisfying for almost all is called an admissible control. Let be the class of all such admissible controls, and for each control , the system (1a)-(1b) has a unique solution .
Consider the following continuous inequality constraints on the control and the state: where is continuously differentiable with respect to all its arguments.
Now, the optimal control problem considered in this paper is stated formally as follows.
Problem (). Given the dynamical system (1a)-(1b), find a control such that the object function is minimized subject to the terminal state equality constraint (1c) and the continuous inequality constraints on the state and control (4), where is continuously differentiable with respect to and is continuously differentiable with respect to all its arguments.
3. Problem Approximation
Let the interval be partitioned into subintervals , with a sequence of time points , satisfying , where , . is denoted as the indictor function in defined by The admissible control is approximated by a piecewise-constant function given below: where , are control parameters, and let denote the set of all such .
In order to further improve the accuracy of the approximate optimal control problem, the switching points are also taken as decision variables. We will employ the time scaling transform originally proposed in [11] to map these switching points, , into preassigned fixed points, , in a new time horizon . This is easily achieved through the following differential equation: with initial condition where Here, , are the control parameters; let be the the set containing all such , and is defined by Taking integration of (8a) and (8b) with initial condition (8b), it is easy to deduce that, for , Clearly, for , In the new time horizon , the approximate control given by (7) becomes where are the fixed switching points.
Let According to (14), the dynamic systems (1a), (1b), (1c), and (8a) can be rewritten as with the initial condition and the terminal condition where Let denote the solution of (16a)-(16b) corresponding to . The inequality constraints on the state and control (4) and the objective function (5) become respectively, where
After the control parametrization and a time scaling transformation, the approximate parameter selection problem corresponding to Problem () may be stated formally as follows.
Problem (). Given system (16a)-(16b), find a control parameter such that the objective function (19) is minimized subject to (16c) and (18).
Problem () is an optimal parameter selection problem in which a finite number of decision variables (the control parameters) need to be optimized subject to a set of constraints. It is very difficult to solve this optimal parameter selection problem because each continuous inequality constraint in (18) actually constitutes an infinite number of constraints one for each point in . In the next section, we will use an exact penalty method to deal with this approximate problem ().
4. A Simple Exact Penalty Function Method
Define the following constraint violation function on : It is clear that if and only if (16c) and (18) are both satisfied. Let be a given constant. By applying the exact function method introduced in [2] to (19) and (21), we construct a simple exact penalty function as follows: where , , , and is a penalty parameter.
From the definition of , it follows that it involves three different cases, only the second case is useful in practical computation. In the second case, it contains the sum of three terms: the first term penalizes system cost, the second term penalizes constraint violations, and the third term penalizes large values of . As the penalty parameter is increasing, minimizing forces to be small, which in turn causes to become large. Consequently, constraint violation functions will approach to zero. In this way, the satisfaction of the equality constraints (16c) and the continuous inequality constraints (18) will eventually be achieved. Thus, problem () is equivalent to the following problem, which is denoted as problem (); we can solve it for an increasing sequence of penalty parameters.
Problem (). Given system (16a)-(16b), find parameter such that the new objective function is minimized.
Problem () is much easier to solve than problem (). Numerical algorithms for solving such a problem need to use the gradient of the new objective function (23) to find the optimal solution. In the following, we will rewrite (23) in the canonical form as in [15] to obtain its gradient formulas.
Let From (22), it follows that Now, the objective function of problem () is in canonical form. Based on the proof of Theorem in [15], the gradient formulas of the objective functions (22) are given in the following theorem.
Theorem 1. The gradients of the objective function with respect to , , and are respectively, where is the Hamiltonian function for the objective function (25) given by and is the solution of the following system of costate differential equations: with the boundary condition
In particular, when , we have . Thus, a corresponding gradient formula of the objective function can also be obtained.
In the following, we will turn our attention to the new objective function given by (22). We will see that, under some mild conditions, is continuously differentiable with respect to all its arguments.
Theorem 2. Suppose that is a local minimizer of problem and that is finite. If then,
Proof. First, we will show that (32a) holds.
Following arguments similar to those given for the proof of Lemma in [15], we obtain
From (19), (33), and the proof of Lemma in [15], it follows that
Since () is a local minimizer of problem (), it is deduced that
That is,
Thus, we have
This, together with (22) and (34), yields (32a).
Next, from the proof process given for (32a), it is easy to conclude that (32b) holds.
Furthermore, we will move on to show that (32c) holds.
From (30a) of Theorem 1, we obtain
In view of (30b) and (37), it follows that
By Schwarz inequality, we have
This, together with (32b) and (37), yields
Since is continuously differentiable with respect to all its arguments, there exists a constant such that
By using (41) and (42), we obtain
Thus,
Similar to obtaining (42), there exist constants and such thatThus, by applying Gronwall-Bellman’s lemma (see Theorem in [15]) to (38), it follows from (39), (44), (45a), and the Lebesgue dominated convergence theorem (see Theorem in [15]) that
By arguments similar to those used to obtain (44), it is easy to show that
From (37) and (45b), we obtain
Finally, by argument similar to those given for (32c), we can show that
This means that (32d) holds. Thus, the proof is completed.
5. Convergence Analysis
In this section, we will show that if () is a local minimizer of problem () and the parameter is sufficiently large as , then and with () being a local minimizer of problem ().
Lemma 3. Suppose that is a local minimizer of problem , that is finite, and that . If there exist , and a sufficiently large number such that then .
Proof. On the contrary, we assume that the conclusion is false. Then, there exists a subsequence of , which is still denoted by the original sequence, such that . On one hand, by using the fact that is continuously differentiable with respect to all its arguments, it follows from the definition of and Lemma in [15] that for and , where is a given constant. From (21) and (51), there exists a constant such that for all . In particular, for , it is clear that On the other hand, since () is a local minimizer of problem (), it follows that This implies that Choose Then, This is a contradiction to (52). The proof is completed.
Lemma 4. Suppose that is a local minimizer of problem , that is finite, and that . Then, .
Proof. On the contrary, we assume that the conclusion is false. Then, . By (28) and the assumption that , we obtain On one hand, () is a local minimizer of problem () and . Thus, This is a contradiction to (57). Hence, . The proof is completed.
Theorem 5. Suppose that is a local minimizer of problem such that and is finite. If there exits a sufficiently large number such that then and .
Proof. By Lemma 4, we have . In the following, we will show that . Since () is a local minimizer of problem () and , it follows that
That is,
It is clear that
By using (59), (62), and Lemma 3, we obtain . Therefore, .
Applying (62) and Lemma 3 to (22), it follows that
The proof is completed.
Theorem 6. Suppose that is a local minimizer of problem such that and is finite. If there exist and a sufficiently large number such that then and is a local minimizer of problem ().
Proof. Since () is a local minimizer of problem (), it follows from Lemma 3 that . This means that is also a local minimizer of problem (). This, together with Theorem 5, implies that () is a local minimizer of problem (). The proof is completed.
Once the optimal parameter of problem () is determined, the corresponding optimal control can be obtained readily from (7) and (12).
Theorem 7. Let be an optimal control of the approximate problem . Then, where is an optimal control of the problem ().
Proof. The proof is similar to that given for Theorem in [15].
Based on the results of the convergence analysis, we are in a position to present the following numerical algorithm for solving problem ().
Algorithm 8. The steps are as follows.
Step 1. Set , , , choose an initial point , and the iteration index . The values of and are chosen depending on the specific structure of the concerned problem , where and .
Step 2. Solve problem , and let be the minimizer obtained.
Step 3. If , , set , , and go to Step 2 with as the new initial point in the new optimization process. Otherwise, set and . is a solution of problem .
Remark 9. In Step 3, if , it follows from Lemmas 3 and 4, and Theorem 5 that cannot be a feasible point. This means that the penalty parameter is not chosen large enough. Thus, we need to increase . If , but still , then we should choose a different initial condition or adjust the values of and such that the conditions of Theorem 2 are satisfied. Then, set , , . Go to Step 2.
Remark 10. Although we have proven that a local minimizer of the exact penalty function optimization problem will converge to a local minimizer of the original problem , we need, in actual computation, to set a lower bound for , so as to avoid the situation of being divided by , leading to infinity.
6. Simulation Results
Example 1. In this example, we consider a realistic and complex problem of transferring containers from a ship to a cargo truck at the port of Kobe in [1]. The container crane is driven by a hoist motor and a trolley drive motor. In order to ensure safety, our objective is to minimize the swing during and at the end of the transfer. After appropriate normalization, this problem is summarized as follows: find controls and such that the object function is minimized subject to dynamical equations as follows: where the initial and terminal conditions are and the control and state inequality constraints are respectively. Furthermore, the state inequality constraints can be formulated as follows:
By utilizing the control parametrization method used conjunction with a time scaling transform and the exact penalty function method, the constrained optimal control problem is transformed into the following unconstrained optimal control problem (), given the following system: with the initial condition . Find control and system parameters such that the new object function is minimized, where Thus, set , , and . Applying Algorithm 8 to problem (), the obtained results are shown below. The optimal objective function is , where and . The optimal controls, the optimal states, and the constraint functions are shown in Figures 2, 3, 4, and 5, respectively. From Figure 1, we observe that the obtained optimal controls are satisfied for the constraint (69). Under the optimal controls, the terminal state equality constraint (68) and the state inequality constraint (71) are both satisfied, which are seen from Figures 2 to 4, respectively. Comparing with the results of Example in [15], our minimum objective function value is slightly larger than one in [15] (it is in [15]). However, in [15], the state inequality constraints (71) are not completely satisfied for all .
Example 2. The following problem is taken from [16]. Find a control that minimizes the objective function subject to dynamic equations with initial condition and the continuous inequality constraint on the control and state
In this problem, we set , , and . Similar to solve Example 1, the obtained result is shown below. The optimal objective function value is . where and . The optimal control, the optimal states, and the constraint function are shown in Figures 5, 6, 7, and 8, respectively. From Figure 8, we observe that the control and state constraints are satisfied for all .
7. Conclusions
In this paper, we have presented optimal control problems subject to the terminal state equality constraint and continuous inequality constraints on the control and the state. Our aim is to design an optimal control that minimizes total system cost and ensures satisfaction of all constraints. After the control parametrization, together with the time scaling transformation, the constrained optimal control problem is transformed into a constrained approximate optimal parameter selection problem. A simple exact penalty function method is then used to design a computational method to solve the constrained optimal parameter selection problem. Its main idea is to augment the constraint violation function constructed from the terminal state equality constraint and continuous inequality constraints to the objective function, forming a new one. This gives rise to a sequence of unconstrained optimal control problems, which are easily solved by a numerical algorithm. From numerical simulation results, we observe that our proposed method can find a high quality approximate optimal control such that the objective function is minimized, while the terminal state constraint and the constraints on the control and state are both satisfied.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This work was supported by the National Natural Science Foundation of China under Grant no. 11371006.