About this Journal Submit a Manuscript Table of Contents
Mathematical Problems in Engineering
Volume 2012 (2012), Article ID 581040, 22 pages
http://dx.doi.org/10.1155/2012/581040
Research Article

Optimality Condition-Based Sensitivity Analysis of Optimal Control for Hybrid Systems and Its Application

State Key Laboratory of Industrial Control Technology, Institute of Industrial Process Control, Zhejiang University, Hangzhou 310027, China

Received 4 April 2012; Revised 21 June 2012; Accepted 11 July 2012

Academic Editor: Jun Hu

Copyright © 2012 Chunyue Song. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Gradient-based algorithms are efficient to compute numerical solutions of optimal control problems for hybrid systems (OCPHS), and the key point is how to get the sensitivity analysis of the optimal control problems. In this paper, optimality condition-based sensitivity analysis of optimal control for hybrid systems with mode invariants and control constraints is addressed under a priori fixed mode transition order. The decision variables are the mode transition instant sequence and admissible continuous control functions. After equivalent transformation of the original problem, the derivatives of the objective functional with respect to control variables are established based on optimal necessary conditions. By using the obtained derivatives, a control vector parametrization method is implemented to obtain the numerical solution to the OCPHS. Examples are given to illustrate the results.

1. Introduction

In many fields of applications, such as powertrain systems of automobiles and multistage chemical processes, dynamics of the systems involve a sequence of distinct modes with fixed mode transition order, forming a hybrid system characterized by the coexistence and interaction of discrete and continuous dynamics (the mode is commonly denoted by a discrete state of the systems in hybrid systems literature). To achieve some overall optimal performance for the systems, the duration and the admissible continuous control function of each mode must be determined as a whole [13]; thus, it necessitates the use of theories and techniques for the analysis and synthesis of hybrid dynamical systems. With the growing importance of hybrid models, various classes of hybrid systems for analysis, design, and optimization have been addressed by research communities in recent years. For more discussions on various literature results, the reader is referred to [48], and the references therein.

The existed results on OCPHS can be divided into the following two categories. One is about the optimal control theory on OCPHS. The theory inherits conventional optimal control theory and can be regarded as the extension of conventional optimal control theory [3, 914]. When control can take any value, Xu and Antsaklis [3] and Hwang et al. [9] addressed the variational method for hybrid systems. Sussmann [10], Shaikh and Caines [11], and Dmitruk and Kaganovich [12] established the Maximum Principle for hybrid systems with control constraints. Branicky et al. [14] and Bensoussan and Menaldi [13] provided the dynamic programming principle for general hybrid systems.

The other results focus on how to compute optimal control for OCPHS, which can be carried out by using a wide variety of methods (see [3, 6, 11, 1520] and the references therein). Given a prespecified order of mode transitions, Xu and Antsaklis [3] obtained the optimal continuous control and optimal switching instants based on parameterization of the switching instant for switching hybrid systems with free control. Under a fixed switching sequence of modes, Attia et al. [19] considered an optimization problem for a class of impulsive hybrid systems where continuous control function is not involved. When switching hybrid systems with control constraints are considered, Shaikh and Caines [11] proposed two algorithms for obtaining the optimal control. As far as switching hybrid systems without external continuous control function are concerned, Egerstedt et al. [6] and Johnson and Murphey [18] derived the gradients and second-order derivatives of the cost functional, respectively, and used them to design an associated algorithm to get the mode transition instants. Based on the hybrid Maximum Principle, Taringoo and Caines [20] provided gradient geodesic and Newton geodesic algorithms for the optimization of autonomous hybrid systems, and convergence analysis for the algorithms was also provided. From the view of dynamic programming, Seatzu et al. [16] provided an optimal state feedback control law to switched piecewise affine autonomous systems. Generally, these algorithms pose the hierarchy [17, 21, 22], and the basic module of the hierarchical algorithms is how to get optimal continuous control and optimal mode transition instants, though the main challenge of OCPHS is how to get the optimal mode transition order. The basic module of the hierarchical algorithms is commonly gradient based due to that gradient information can provide a better searching direction and hence reduce computation burden and help the gradient-based algorithms converge quickly, which motivates us to pay attention to the sensitivity analysis of optimal control for hybrid systems.

Although the derivative of cost functional with respect to switching instants has been discussed in the aforementioned literature [3, 6, 18], the derivative of cost functional with respect to control function is not involved. When hybrid systems are considered, due to the coexistence and interaction of discrete and continuous dynamics, the derivative of cost functional w.r.t control functions is nontrivial and is not directly formulated by 𝜕𝐻/𝜕𝑢 as conventional optimal control indicates, where 𝐻 is the Hamiltonian function. The derivative will be a function of the derivatives of continuous states w.r.t control functions at the instants of subsequent modes. In this paper, the derivatives of cost functional w.r.t control functions are established analytically, which can facilitate the design of associated gradient-based algorithms.

Motivated by the work of Vassiliadis et al. [1, 2] and Jennings et al. [23], in this paper, optimal control problem of hybrid systems (OCPHS) with mode invariants which describe the conditions that continuous states have to satisfy at this mode are considered. Based on optimal necessary conditions, the derivatives of the objective functional w.r.t control variables, that is, the mode transition instant sequence and admissible continuous control functions, are derived analytically. As a result, a control vector parametrization method is implemented to obtain the numerical solution to optimal control of the hybrid systems with the obtained derivatives. The sensitivity analysis in Vassiliadis et al. [1, 2] is similar to the work, in which the sensitivity of states w.r.t control parameters is directly obtained from the state equations and the sensitivity of objective functional with respect to control parameters is not involved. In contrast, this paper derives the derivatives of cost functional w.r.t control variables based on the optimality conditions and gives the explicitly expression of the derivatives. Therefore, the main contributions of this paper are listed as follows. (a) Optimality conditions-based sensitivity analysis of optimal control for hybrid systems with mode invariants are given explicitly, and (b) following the given derivatives, a control vector parameterization method is designed to obtain the numerical solution. Compared with the existing results on the OCPHS with fixed mode transition order, the settings in this paper cover not only the control constraints, but also the continuous states constraints, which makes the results here more general.

The paper is organized as follows. In the next section, the hybrid system with mode invariants and its optimal control problem are formulated. In Section 3, the equivalent problem and associated optimal conditions are analyzed. The derivatives of the objective functional w.r.t control variables are established in Section 4, and a control vector parametrization approach is also proposed in this section. Some numerical examples are presented in Section 5, and Section 6 contains conclusions.

Terminology and Notation
denotes the set of positive integers. and + denote the set of real numbers and nonnegative real numbers, respectively. 𝐴𝑇 denotes the transpose of a vector (or a matrix) 𝐴. 𝐶𝑙([𝑎,𝑏],𝑛) denotes the family of continuous functions 𝑓 from [𝑎,𝑏] to 𝑛 with up to 𝑙 order derivatives. denotes the Euclidean norm.

2. Hybrid Systems and Its Optimal Control Problem

2.1. Hybrid Systems

Engineered systems, such as chemical engineering systems and powertrain systems of automobiles, always undergo multiple modes which are represented by a discrete state 𝑖 taking values from set 𝐼{1,2,,𝑀} and pose hybrid characters. The evolution of discrete state 𝑖 is determined by mode transition sequence. A mode transition sequence schedules the sequence of active modes 𝑖𝑗,𝑖𝑗𝐼 and is a sequence of pairs of (𝑡𝑗1,𝑖𝑗), which can be defined by {(𝑡0,𝑖1),(𝑡1,𝑖2),}(𝜃,𝜋) where 𝜃{𝑡0,𝑡1,} and 𝜋{𝑖1,𝑖2,} are referred to as mode transition instants and mode transition order, respectively. A pair of (𝑡𝑗1,𝑖𝑗) indicates that at instant 𝑡𝑗1, the hybrid system transits from mode 𝑖𝑗1 to mode 𝑖𝑗. During the time interval [𝑡𝑗1,𝑡𝑗), mode 𝑖𝑗 is active and unchanged.

The mode transition order 𝜋 of the considered hybrid dynamical systems is known a priori. Without loss of generality, it is supposed that the mode transition order is {𝑖1,𝑖2,,𝑖𝐾} over the finite horizon [𝑡0,𝑡𝑓], 𝑖𝑗𝐼, 𝑗=1,2,,𝐾. Moreover, according to each distinct mode, the continuous states are restricted in a specified range which is referred to as mode invariants. Here, the mode invariants are formulated by a set of inequalities. Thus, for each mode 𝑖𝑗𝐼 and its active horizon [𝑡𝑗1,𝑡𝑗), the dynamics of the considered systems can be formulated by ̇𝑥=𝑓𝑖𝑗𝑝(𝑥,𝑢),𝑖𝑗𝑥𝑡(𝑥)<0,𝑗1=𝜓𝑖𝑗𝑥𝑡𝑗1,g𝑖𝑗𝑥𝑡𝑗=0,(2.1) where 𝑥𝑛, 𝑢𝐔𝑖𝑗𝑚 is a piecewise continuous function, 𝑓𝑖𝑗𝑛×𝐔𝑖𝑗𝑛, 𝑡𝑗 is the mode transition instant when a particular mode transition occurs, 𝑝𝑖𝑗, 𝜓𝑖𝑗, and 𝑔𝑖𝑗 are 𝑖𝑗<𝑛, 𝑛 and 𝑟𝑖𝑗𝑛 dimensional vectors for 𝑖𝑗𝐼, respectively. 𝑛,𝑚,𝑖𝑗,𝑟𝑖𝑗. To make the hybrid systems formulated by (2.1) well defined, the following assumption is needed.

Assumption 2.1. For any 𝑖𝑗𝐼, 𝑓𝑖𝑗𝐶𝑙(𝑛×𝐔𝑖𝑗;𝑛),𝑙1,𝑙, and such that a uniform Lipschitz condition holds, that is, there exists 𝐾𝑓< such that 𝑓𝑖𝑗(𝑥,𝑢)𝑓𝑖𝑗𝑥,𝑢𝐾𝑓𝑥𝑥,(2.2) where 𝑥,𝑥𝑛,𝑢𝐔𝑖𝑗.

Remark 2.2. 𝑝𝑖𝑗(𝑥)<0 indicates mode invariant for mode 𝑖𝑗𝐼, which describes the conditions that the continuous states have to satisfy at this mode and can be referred to as the path constraints of the continuous states in Vassiliadis et al. [1, 2].

Remark 2.3. 𝑔𝑖𝑗(𝑥(𝑡𝑗))=0 can be referred to as mode transition conditions which describe the conditions on the continuous states under which a particular mode transition takes place. When mode 𝑖𝑗 is active over [𝑡𝑗1,𝑡𝑗), then, at 𝑡𝑗, 𝑥 meets an (𝑛𝑟𝑖𝑗)-dimensional smooth manifold 𝑆𝑖𝑗={𝑥𝑔𝑖𝑗(𝑥)=0} and mode transition from 𝑖𝑗 to 𝑖𝑗+1 occurs. The mode transition conditions implicitly define the mode 𝑖𝑗's active horizon [𝑡𝑗1,𝑡𝑗). To prevent Zeno behavior from occurrence, 𝑡𝑗1<𝑡𝑗 is assumed. Physically, the mode transition conditions are always the boundary of closure of the mode invariant 𝑝𝑖𝑗<0.

Remark 2.4. 𝑥(𝑡𝑗1)=𝜓𝑖𝑗(𝑥(𝑡𝑗1)) is the outcome of the mode transition and describes the effect that the transition will have on the continuous states. It can be viewed as junction conditions in Vassiliadis et al. [1, 2]. It is assumed that 𝜓𝑖𝑗𝐶𝑙(𝑛), 𝑙1, 𝑙.

Remark 2.5. Basically, for general hybrid systems, the evaluation of 𝑖 should be formulated by a function of impulsive control or a graph, which generates mode transition sequence, as formulated in Song and Li [24] and Cassandras and Lygeros [8]. However, the order of the mode transition 𝜋 is known a prior here thus, the evaluation of 𝑖 is determined only by the transition instants 𝑡𝑗, and the evaluation function of 𝑖 is omitted here.
Besides Assumption 2.1, to make the considered systems to be well defined, there are some additional assumptions on mode invariants and mode transition conditions should be imposed. Here, it is supposed that the mode invariants and mode transition conditions meet the requirements as in Taringoo and Caines [20].

2.2. Optimal Control Problem for Hybrid Systems

Let 𝐿𝑖𝐶𝑙(𝑛×𝐔𝑖;) be a running cost function, 𝜑𝑖𝑗𝐶𝑙(𝑛;+) be a discrete state transition cost function, and 𝜙𝐶𝑙(𝑛;+) be a terminal cost function, 𝑖,𝑗𝐼, 𝑙1, 𝑙, respectively. The optimal control problem for the hybrid systems (2.1) is stated as follows.

Optimal Problem A
Consider a hybrid system formulated by (2.1), given a fixed time interval [𝑡0,𝑡𝑓] and a prespecified mode transition order 𝜋={𝑖1,𝑖2,,𝑖𝐾}, find a continuous control 𝑢𝐔𝑖𝑗 in each mode 𝑖𝑗𝐼 and mode transition instants 𝜃={𝑡1,,𝑡𝐾1}, such that the corresponding continuous state trajectory 𝑥 departs from a given initial state 𝑥(𝑡0)=𝑥0 and meets an (𝑛𝑙𝑓)-dimensional smooth manifold 𝑆𝑓={𝑥𝜗(𝑥)=0,𝜗𝑛𝑙𝑓}, 𝑙𝑓, at 𝑡𝑓 and the cost functional 𝑥𝑡𝐽(𝜃,𝑢)=𝜙𝑓+𝑡𝑓𝑡0𝐿𝑖(𝑡)(𝑥(𝑡),𝑢(𝑡))𝑑𝑡+𝐾1𝑗=1𝜑𝑖𝑗𝑖𝑗+1𝑥𝑡𝑗(2.3) is minimized.

Remark 2.6. As it is well known, when 𝑡0 and 𝑡𝑓 are unknown points in some fixed interval 𝑇+, this problem can be transformed to one with fixed time essentially by introducing an additional state variable.

There are fruitful strategies about how to compute OCPHS (see [15] and the references therein), and the basic idea is briefly reviewed as follows for completeness.

Obtaining the optimal control for hybrid systems is very difficult due to the interactions between the continuous states and discrete states which produce a mode transition sequence that increases the feasibility range of the decision variables. One algorithm framework for dealing with this complexity is the decomposition method as follows: min((𝜋,𝜃),𝑢)𝐽((𝜋,𝜃),𝑢)=min(𝜋,𝜃)min𝑢𝐽(𝑢(𝜋,𝜃))=min𝜋min𝜃min𝑢𝐽((𝑢,𝜃)𝜋),(2.4) where 𝐽(𝑏) means that 𝑏 is given.

According to this framework, the master problem is how to get the optimum of the inner functional, that is, minimize 𝐽(𝑢,𝜃) given 𝜋. The key point of finding the optimal solution of 𝐽(𝑢,𝜃) is how to get the sensitivity of the objective with respect to control variables, which provides a better direction for searching and hence reduces computational burden and help associated algorithms converge quickly and accelerate the primary problem convergence eventually.

In next section, the derivatives of cost functional with respect to control variables are established analytically based on optimality condition, which can facilitate the design of associated gradient-based algorithms.

3. Equivalent Problem and Its Optimal Conditions

When control vector parametrization methods are implemented to obtain numerical solution to the OCPHS, updating the parameters of control profiles should be at the same time point when iterative procedure is running. However, the fact is that the mode active horizon [𝑡𝑗1,𝑡𝑗) for mode 𝑖𝑗𝐼 is varying during the procedure running, so a fixed horizon should be introduced, which will guarantee the updating of parameters of control profiles is at the same time point. For this purpose, let 𝜏[0,𝐾] be a time independent variable, and 𝑡[𝑡𝑗1,𝑡𝑗) can be formulated by 𝑡=𝑡𝑗1+𝑡(𝜏(𝑗1))𝑗𝑡𝑗1[,𝜏𝑗1,𝑗),𝑗=1,,𝐾.(3.1)

In addition, to deal with mode invariants constraints 𝑝𝑖𝑗(𝑥)<0, slack algebraic variable 𝑠𝑖𝑗=[𝑠𝑖𝑗1,,𝑠𝑖𝑗𝑖𝑗]𝑇𝑖𝑗+ is introduced for each mode 𝑖𝑗𝐼, such that 𝑝𝑖𝑗(𝑥)+diag[𝑠𝑖𝑗1,,𝑠𝑖𝑗𝑖𝑗]𝑠𝑖𝑗=0. For 𝜏[𝑗1,𝑗), denote 𝐱𝑗(𝜏)𝑥(𝑡𝑗1+(𝜏(𝑗1))(𝑡𝑗𝑡𝑗1)), 𝐮𝑗(𝜏)𝑢(𝑡𝑗1+(𝜏(𝑗1))(𝑡𝑗𝑡𝑗1)), 𝐬𝑗(𝜏)𝑠𝑖𝑗(𝑡𝑗1+(𝜏(𝑗1))(𝑡𝑗𝑡𝑗1)), and let 𝐱=[𝐱1,,𝐱𝐾]𝑇, 𝐮=[𝐮1,,𝐮𝐾]𝑇, and 𝐬=[𝐬1,,𝐬𝐾]𝑇.

According to the above definition, the Optimal Problem A can be transcribed into an equivalent Optimal Problem B as follows:

Optimal Problem B
Given a fixed interval [0,𝐾], find continuous inputs 𝐮𝐔𝑖1××𝐔𝑖𝐾, 𝐬𝑖1+××𝑖𝐾+ and 𝜃, such that the corresponding continuous state trajectory 𝐱1 departs from a given initial state 𝐱1(0)=𝑥0 and 𝐱𝐾 meets an (𝑛𝑙𝑓)-dimensional smooth manifold 𝑆𝑓={𝐱𝐾𝜗(𝐱𝐾)=0,𝜗𝑛𝑙𝑓} at 𝐾, and the cost functional 𝐱𝐽(𝜃,𝐮,𝐬)=𝜙𝐾+(𝐾)𝐾𝑗=1𝑗𝑗1𝐿𝑖𝑗𝐱𝑗(𝜏),𝐮𝑗(𝜏),𝐬𝑗(𝜏)𝑑𝜏+𝐾1𝑗=1𝜑𝑖𝑗𝑖𝑗+1𝐱𝑗(𝑗)(3.2) is minimized, subject to 𝑑𝐱𝑗(𝜏)=𝑓𝑑𝜏𝑖𝑗𝐱𝑗(𝜏),𝐮𝑗𝑡(𝜏)𝑗𝑡𝑗1𝑓𝑖𝑗𝐱𝑗(𝜏),𝐮𝑗,𝐱(𝜏)𝑗(𝑗1)=𝜓𝑖𝑗𝐱𝑗1((𝑗1)),𝑔𝑖𝑗𝐱𝑗(𝑗)=0,(3.3) where 𝐿𝑖𝑗𝐱𝑗,𝐮𝑗,𝐬𝑗=𝑡𝑗𝑡𝑗1𝐿𝑖𝑗,𝐿𝑖𝑗=𝐿𝑖𝑗𝐱𝑗,𝐮𝑗+𝑀𝑖𝑗𝑙=1𝑝𝑖𝑗𝑙𝐱𝑗+𝑠2𝑖𝑗𝑙2,(3.4) and 𝑀 is a large positive constant.

According to Theorems 2 and 3 in Dmitruk and Kaganovich [12], when 𝑀 is big enough Optimal Problem B is equivalent to Optimal Problem A.

Remark 3.1. The penalty function term, say, 𝑀𝑖𝑗𝑙=1(𝑝𝑖𝑗𝑙(𝐱𝑗)+𝐬2𝑖𝑗𝑙)2, cannot always guarantee the state satisfies the mode invariant conditions. However, the method works well in practice; moreover, the mode transition order is fixed in this paper which reduces the negative effect of the penalty function method for OCPHS.

For 𝜏[𝑗1,𝑗), 𝑗=1,,𝐾, let 𝜆𝑗𝑛, and define Hamiltonian function 𝐻𝑗 by 𝐻𝑗𝜆𝑗,𝐱𝑗,𝐮𝑗,𝐬𝑗=𝐿𝑖𝑗𝐱𝑗,𝐮𝑗,𝐬𝑗+𝜆𝑇𝑗𝑓𝑖𝑗𝐱𝑗,𝐮𝑗,(3.5) and according to Sussmann [10], Shaikh and Caines [11], and Dmitruk and Kaganovich [12], the following Theorem 3.2 holds.

Theorem 3.2. In order that 𝐮 and 𝐬 are optimal for Optimal Problem B, it is necessary that there exist vector functions 𝜆𝑗, 𝑗=1,,𝐾, such that the following conditions hold:(a)for almost any 𝜏[𝑗1,𝑗), the following state equations hold: 𝑑𝐱𝑗(𝜏)=𝑓𝑑𝜏𝑖𝑗𝐱𝑗(𝜏),𝐮𝑗,(𝜏)(3.6)(b)for almost any 𝜏[𝑗1,𝑗), the following costate equations hold: ̇𝜆𝑗𝜕𝐿=𝑖𝑗𝜕𝐱𝑗𝑇𝜕𝑓𝑖𝑗𝜕𝐱𝑗𝑇𝜆𝑗,(3.7)(c)for a.e. 𝜏[𝑗1,𝑗), 𝐻𝑗𝜆𝑗,𝐱𝑗,𝐮𝑗,𝐬𝑗=0,(3.8)(d)minimality condition: for all 𝜏[𝑗1,𝑗), min𝐮𝑗𝐔𝑖𝑗,𝐬𝑗𝑖𝑗+𝐻𝑗𝜆𝑗,𝐱𝑗,𝐮𝑗,𝐬𝑗=0,(3.9)(e)transversality conditions for 𝜆𝑗, 𝜆𝑗+1(𝑗)=𝛽𝑗𝜆,𝑗=1,,𝐾1,𝑗(𝑗)=𝜕𝑔𝑖𝑗𝜕𝐱𝑗(𝑗)𝑇𝛼𝑗𝜕𝜓𝑖𝑗+1𝜕𝐱𝑗(𝑗)𝑇𝛽𝑗+𝜕𝜑𝑖𝑗𝑖𝑗+1𝜕𝐱𝑗(𝑗)𝑇𝜆,𝑗=1,,𝐾1,𝐾(𝐾)=𝜕𝜙𝜕𝐱𝐾(𝐾)𝑇+𝜕𝜗𝜕𝐱𝐾(𝐾)𝑇𝛼𝐾,(3.10)where 𝛼𝑗𝑖, 𝛽𝑗𝑛 are Lagrangian multipliers. Based on Theorem 3.2, the sensitivity analysis is established in the next section for Optimal Problem B.

4. Sensitivity Analysis and Parametrization Method

For finding numerical solution to the OCPHS effectively, based on Theorem 3.2, the derivatives of the objective functional 𝐽() with respect to the control 𝐮, 𝐬, and the mode transition instant 𝑡𝑗,𝑗=1,,𝐾1 are established in this section, and by using the obtained derivatives associated parametrization method is proposed.

4.1. Sensitivity Analysis

Lemma 4.1. The derivatives of 𝐱𝑗(𝑗),𝑗=1,,𝐾, w.r.t 𝑡𝑘 and 𝐮𝑘 are given, respectively, as follows for 𝑘=1,,𝐾1, 𝑑𝐱𝑗(𝑗)𝑑𝑡𝑘=0,𝑗=1,,𝑘1,𝑑𝐱𝑘(𝑘)𝑑𝑡𝑘=𝑓𝑖𝑘𝐱𝑘(𝑘),𝐮𝑘(𝑘),𝑑𝐱𝑘+1((𝑘+1))𝑑𝑡𝑘=Ω𝑘+1,𝑑𝐱𝑗(𝑗)𝑑𝑡𝑘=𝑗𝑙=𝑘+2Φ𝑙(𝑙,𝑙1)𝑑𝜓𝑖𝑙𝑑𝐱𝑙1((𝑙1))Ω𝑘+1,𝑗=𝑘+2,,𝐾,(4.1)𝛿𝐱𝑗(𝑗)𝛿𝐮𝑘=0,𝑗=1,,𝑘1,𝛿𝐱𝑘(𝑘)𝛿𝐮𝑘=Γ𝑘(𝜏),𝛿𝐱𝑗(𝑗)𝛿𝐮𝑘=𝑗𝑙=𝑘+1Φ𝑙(𝑙,𝑙1)𝑑𝜓𝑖𝑙𝑑𝐱𝑙1((𝑙1))Γ𝑘(𝜏),𝑗=𝑘+1,,𝐾,(4.2) where Ω𝑘+1=Φ𝑘+1(𝑘+1,𝑘)𝑑𝜓𝑖𝑘+1𝑑𝐱𝑘(𝑘)𝑓𝑖𝑘𝐱𝑘(𝑘),𝐮𝑘(𝑘)𝑓𝑖𝑘+1𝐱𝑘+1(𝑘),𝐮𝑘+1,Γ(𝑘)𝑘𝑡(𝜏)=𝑘𝑡𝑘1Φ𝑘(𝑘,𝜏)𝜕𝑓𝑖𝑘𝜕𝐮𝑘,Φ𝑙(𝜏,𝑣)=exp𝜏𝑣𝑡𝑙𝑡𝑙1𝜕𝑓𝑖𝑙𝜕𝐱𝑙.𝑑𝑎(4.3)

Note that 𝐱(𝑡𝑗) is a functional vector of 𝐮𝑘, and the expression 𝛿𝐱𝑗/𝛿𝐮𝑘 is used, where the notation 𝛿𝐱𝑗/𝛿𝐮𝑘 is the functional derivatives which describe the response of the functional 𝐱𝑗 to an infinitesimal change in the function 𝐮𝑘 at each point.

Proof. The proof of (4.1) is only going to be shown for easily reading. The proof for (4.2) can be found in Appendix.
When 𝑗=1,,𝑘1, 𝐱𝑗(𝑗) and 𝐱𝑗+1(𝑗) are independent of 𝑡𝑘, and obviously 𝑑𝐱𝑗(𝑗)/𝑑𝑡𝑘=0 holds. In the case of 𝑗=𝑘, 𝐱𝑘(𝑘) is a function of 𝑡𝑘 which gives rise to 𝑑𝐱𝑘(𝑘)/𝑑𝑡𝑘=𝑓𝑖𝑘(𝐱𝑘(𝑘),𝐮𝑘(𝑘)).
Case i. (𝑗=𝑘+1). In this case, 𝐱𝑘+1 is a function of 𝑡𝑘 and 𝐱𝑘+1(𝑘), and we have 𝑑𝐱𝑘+1(𝜏)𝑑𝑡𝑘=𝜕𝐱𝑘+1𝜕𝑡𝑘+𝜕𝐱𝑘+1𝜕𝐱𝑘+1(𝑘)𝜕𝐱𝑘+1(𝑘)𝜕𝑡𝑘.(4.4)
Note that in (4.4), 𝜕𝐱𝑘+1/𝜕𝑡𝑘 is produced by the perturbation of 𝑡𝑘, and (𝜕𝐱𝑘+1/𝜕𝐱𝑘+1(𝑘))(𝜕𝐱𝑘+1(𝑘)/𝜕𝑡𝑘) is produced by the perturbation of 𝐱𝑘+1(𝑘) with respect to 𝑡𝑘. Obviously, for 𝜏[𝑘,𝑘+1), 𝜕𝐱𝑘+1(𝜏)𝜕𝑡𝑘=𝑓𝑖𝑘+1𝐱𝑘+1(𝑘),𝐮𝑘+1.(𝑘)(4.5)
The solution to 𝜕𝐱𝑘+1(𝜏)/𝜕𝐱𝑘+1(𝑘) is given by 𝜕𝐱𝑘+1(𝜏)𝜕𝐱𝑘+1𝑘+𝑡=𝐼+𝑘+1𝑡𝑘𝜏𝑘𝜕𝑓𝑖𝑘+1𝜕𝐱𝑘+1𝜕𝐱𝑘+1(𝑣)𝜕𝐱𝑘+1(𝑘)𝑑𝑣.(4.6)
Equation (4.6) is a linear system about 𝜕𝐱𝑘+1/𝜕𝐱𝑘+1(𝑘). Define the state transition matrix Φ𝑙(𝜏,𝑣) by Φ𝑙(𝜏,𝑣)=exp𝜏𝑣𝑡𝑙𝑡𝑙1𝜕𝑓𝑖𝑙(𝑎)𝜕𝐱𝑙(,𝑎)𝑑𝑎(4.7) according to (4.6), and we have 𝜕𝐱𝑘+1(𝜏)𝜕𝐱𝑘+1(𝑘)=Φ𝑘+1(𝜏,𝑘).(4.8)
Thus, 𝑑𝐱𝑘+1(𝜏)𝑑𝑡𝑘=Φ𝑘+1(𝜏,𝑘)𝜕𝐱𝑘+1(𝑘)𝜕𝑡𝑘𝑓𝑖𝑘+1𝐱𝑘+1(𝑘),𝐮𝑘+1.(𝑘)(4.9)
At transition instants 𝑡𝑗, since 𝐱𝑗+1(𝑗)=𝜓𝑖𝑗+1(𝐱𝑗(𝑗)), so 𝑑𝐱𝑗+1(𝑗)𝑑𝑡𝑘=𝑑𝜓𝑖𝑗+1𝑑𝐱𝑗(𝑗)𝑑𝐱𝑗(𝑗)𝑑𝑡𝑘,(4.10) which implies 𝜕𝐱𝑘+1(𝑘)𝜕𝑡𝑘=𝑑𝜓𝑖𝑘+1𝑑𝐱𝑘(𝑘)𝑑𝐱𝑘(𝑘)𝑑𝑡𝑘=𝑑𝜓𝑖𝑘+1𝑑𝐱𝑘(𝑘)𝑓𝑖𝑘𝐱𝑘(𝑘),𝐮𝑘(𝑘).(4.11)
According to (4.9), and we have 𝑑𝐱𝑘+1((𝑘+1))𝑑𝑡𝑘=Φ𝑘+1(𝑘+1,𝑘)𝑑𝜓𝑖𝑘+1𝑑𝐱𝑘(𝑘)𝑓𝑖𝑘𝐱𝑘(𝑘),𝐮𝑘(𝑘)𝑓𝑖𝑘+1𝐱𝑘+1(𝑘),𝐮𝑘+1(𝑘)Ω𝑘+1.(4.12)
Case ii. (𝑗=𝑘+2,,𝐾). When 𝑗=𝑘+2,,𝐾, the following holds: 𝑑𝐱𝑗(𝜏)𝑑𝑡𝑘=𝑑𝐱𝑗(𝑗1)𝑑𝑡𝑘+𝑡𝑗𝑡𝑗1𝜏𝑗1𝜕𝑓𝑖𝑗𝜕𝐱𝑗𝑑𝐱𝑗(𝑣)𝑑𝑡𝑘[𝑑𝑣,𝜏𝑗1,𝑗).(4.13)
Then, 𝑑𝐱𝑗(𝜏)𝑑𝑡𝑘=Φ𝑗(𝜏,𝑗1)𝑑𝐱𝑗(𝑗1)𝑑𝑡𝑘.(4.14)
Substituting the term 𝑑𝐱𝑗(𝑗1)/𝑑𝑡𝑘 in (4.14) by (4.10), we obtain 𝑑𝐱𝑗(𝑗)𝑑𝑡𝑘=𝑗𝑙=𝑘+2Φ𝑙(𝑙,𝑙1)𝑑𝜓𝑖𝑙𝑑𝐱𝑙1((𝑙1))Ω𝑘+1.(4.15)

Theorem 4.2. The derivatives of the objective functional 𝐽() w.r.t 𝑡𝑘, 𝐮𝑘 and 𝐬𝑘 are given, respectively, as follows: 𝑑𝐽𝑑𝑡𝑘=𝐿𝑖𝑘𝐱𝑘(𝑘),𝐮𝑘(𝑘),𝐬𝑘(𝑘)𝐿𝑖𝑘+1𝐱𝑘+1(𝑘),𝐮𝑘+1(𝑘),𝐬𝑘+1(𝑘)+𝜆𝑘(𝑘)𝑇𝑓𝑖𝑘𝐱𝑘(𝑘),𝐮𝑘(𝑘)𝜆𝑘+1(𝑘)𝑇𝑓𝑖𝑘+1𝐱𝑘+1(𝑘),𝐮𝑘+1(𝑘)𝐾1𝑗=𝑘𝛼𝑇𝑗𝜕𝑔𝑖𝑗𝜕𝐱𝑗(𝑗)𝑑𝐱𝑗(𝑗)𝑑𝑡𝑘𝛼𝑇𝐾𝜕𝜗𝜕𝐱𝐾(𝐾)𝑑𝐱𝐾(𝐾)𝑑𝑡𝑘𝛿𝐽𝛿𝐮𝑘=𝜕𝐻𝑘𝜕𝐮𝑘𝐾1𝑗=k𝛼𝑇𝑗𝜕𝑔𝑖𝑗𝜕𝐱𝑗(𝑗)𝛿𝐱𝑗(𝑗)𝛿𝐮𝑘𝛼𝑇𝐾𝜕𝜗𝜕𝐱𝐾(𝐾)𝛿𝐱𝐾(𝐾)𝛿𝐮𝑘𝛿𝐽𝛿𝐬𝑘=𝜕𝐻𝑘𝜕𝐬𝑘.(4.16)

Before proving Theorem 4.2, Lemma 4.3 is firstly given as follows.

Lemma 4.3. For 𝑗=𝑘+2,,𝐾, 𝑑𝑑𝑡𝑘𝑗𝑗1𝐿𝑖𝑗𝐱𝑗,𝐮𝑗,𝐬𝑗𝑑𝜏=𝜆𝑗(𝑗1)𝑇𝑑𝐱𝑗(𝑗1)𝑑𝑡𝑘𝜆𝑗(𝑗)𝑇𝑑𝐱𝑗(𝑗)𝑑𝑡𝑘.(4.17)

Proof. For any 𝑗=𝑘+2,,𝐾, we have 𝑑𝑑𝑡𝑘𝑗𝑗1𝐿𝑖𝑗𝐱𝑗,𝐮𝑗,𝐬𝑗𝑑𝜏=𝑗𝑗1𝑑𝑑𝑡𝑘𝐻𝑗𝜆𝑗,𝐱𝑗,𝐮𝑗,𝐬𝑗𝜆𝑇𝑗𝑓𝑖𝑗=𝑑𝜏𝑗𝑗1𝜕𝐻𝑗𝜕𝐱𝑗𝑑𝐱𝑗𝑑𝑡𝑘+𝜕𝐻𝑗𝜕𝜆𝑗𝑑𝜆𝑗𝑑𝑡𝑘𝑑𝜆𝑗𝑑𝑡𝑘𝑇𝑓𝑖𝑗𝜆𝑇𝑗𝑑𝑑𝑡𝑘𝑓𝑖𝑗𝑑𝜏.(4.18)
Since the following holds by Theorem 3.2, 𝜕𝐻𝑗𝜕𝐱𝑗𝑇̇𝜆=𝑗,𝜕𝐻𝑗𝜕𝜆𝑗𝑇=𝑓𝑖𝑗,(4.19) then 𝑑𝑑𝑡𝑘𝑗𝑗1𝐿𝑖𝑗𝐱𝑗,𝐮𝑗,𝐬𝑗𝑑𝜏=𝑗𝑗1̇𝜆j𝑇𝑑𝐱𝑗𝑑𝑡𝑘+𝑓𝑖𝑗𝑇𝑑𝜆𝑗𝑑𝑡𝑘𝑑𝜆𝑗𝑑𝑡𝑘𝑇𝑓𝑖𝑗𝜆𝑇𝑗𝑑𝑑𝑡𝑘𝑓𝑖𝑗=𝑑𝜏𝑗𝑗1̇𝜆𝑗𝑇𝑑𝐱𝑗𝑑𝑡𝑘𝜆𝑇𝑗𝑑𝑑𝑡𝑘𝑓𝑖𝑗𝑑𝜏=𝑗𝑗1𝑑𝜆𝑑𝜏𝑇𝑗𝑑𝐱𝑗𝑑𝑡𝑘𝑑𝜏=𝜆𝑗(𝑗1)𝑇𝑑𝐱𝑗(𝑗1)𝑑𝑡𝑘𝜆𝑗(𝑗)𝑇𝑑𝐱𝑗(𝑗)𝑑𝑡𝑘.(4.20)

Obviously, when 𝑗=𝑘,𝑘+1, we have 𝑑𝑑𝑡𝑘𝑘𝑘1𝐿𝑖𝑘𝐱𝑘,𝐮𝑘,𝐬𝑘𝑑𝑑𝜏=𝑑𝑡𝑘𝑡𝑘𝑡𝑘1𝐿𝑖𝑘𝑥,𝑢,𝑠𝑖𝑘𝑑𝑡=𝐿𝑖𝑘𝐱𝑘(𝑘),𝐮𝑘(𝑘),𝐬𝑘(𝑘),𝑑(4.21)𝑑𝑡𝑘𝑘𝑘+1𝐿𝑖𝑘+1𝐱𝑘+1,𝐮𝑘+1,𝐬𝑘+1𝑑𝜏=𝜆𝑘+1(𝑘)𝑇𝑑𝐱𝑘+1(𝑘)𝑑𝑡𝑘𝜆𝑘+1((𝑘+1))𝑇𝑑𝐱𝑘+1((𝑘+1))𝑑𝑡𝑘𝐿𝑖𝑘+1𝐱𝑘+1(𝑘),𝐮𝑘+1(𝑘),𝐬𝑘+1(.𝑘)(4.22)

Now we prove Theorem 4.2. We are only going to show 𝑑𝐽/𝑑𝑡𝑘 for easily reading. The proofs for 𝛿𝐽/𝛿𝐮𝑘 and 𝛿𝐽/𝛿𝐬𝑘 can be found in Appendix.

Proof. 𝐽(𝜃,𝐮,𝐬) can be formulated as 𝐱𝐽(𝜃,𝐮,𝐬)=𝜙𝐾+(𝐾)𝑘1𝑗=1𝑗𝑗1𝐿𝑖𝑗𝐱𝑗,𝐮𝑗,𝐬𝑗+𝑑𝜏𝐾𝑗=𝑘𝑗𝑗1𝐿𝑖𝑗𝐱𝑗,𝐮𝑗,𝐬𝑗𝑑𝜏+𝐾1𝑗=1𝜑𝑖𝑗𝑖𝑗+1𝐱𝑗(𝑗).(4.23)
Since 𝐿𝑖𝑗() is independent of 𝑡𝑘 for 𝑗=1,,𝑘1, then 𝑑𝐽/𝑑𝑡𝑘 can be obtained by 𝑑𝐽𝑑𝑡𝑘𝐱(𝜃,𝐮,𝐬)=𝜕𝜙𝐾(𝐾)𝜕𝐱𝐾(𝐾)𝑑𝐱𝐾(𝐾)𝑑𝑡𝑘+𝑑𝑑𝑡𝑘𝐾𝑗=𝑘𝑗𝑗1𝐿𝑖𝑗𝐱𝑗,𝐮𝑗,𝐬𝑗𝑑𝜏+𝐾1𝑗=1𝜕𝜑𝑖𝑗𝑖𝑗+1𝜕𝐱𝑗(𝑗)𝑑𝐱𝑗(𝑗)𝑑𝑡𝑘.(4.24)
Substituting (4.17), (4.21), and (4.22) into (4.24), we have 𝑑𝐽𝑑𝑡𝑘𝐱(𝜃,𝐮,𝐬)=𝜕𝜙𝐾(𝐾)𝜕𝐱𝐾(𝐾)𝑑𝐱𝐾(𝐾)𝑑𝑡𝑘+𝐿𝑖𝑘𝐱𝑘(𝑘),𝐮𝑘(𝑘),𝐬𝑘(𝑘)𝐿𝑖𝑘+1𝐱𝑘+1(𝑘),𝐮𝑘+1(𝑘),𝐬𝑘+1(𝑘)+𝜆𝑘+1(𝑘)𝑇𝑑𝐱𝑘+1(𝑘)𝑑𝑡𝑘+𝜕𝜑𝑖𝑘𝑖𝑘+1𝜕𝐱𝑘(𝑘)𝑑𝐱𝑘(𝑘)𝑑𝑡𝑘𝐾1𝑗=𝑘+1𝜆𝑗(𝑗)𝑇𝑑𝐱𝑗(𝑗)𝑑𝑡𝑘𝜆𝑗+1(𝑗)𝑇𝑑𝐱𝑗+1(𝑗)𝑑𝑡𝑘𝜕𝜑𝑖𝑗𝑖𝑗+1𝜕𝐱𝑗(𝑗)𝑑𝐱𝑗(𝑗)𝑑𝑡𝑘𝜆𝐾(𝐾)𝑇𝑑𝐱𝐾(𝐾)𝑑𝑡𝑘.(4.25)
Due to Theorem 3.2 and (4.10), 𝑑𝐽/𝑑𝑡𝑘 can be formulated by 𝑑𝐽𝑑𝑡𝑘𝐱(𝜃,𝐮,𝐬)=𝜕𝜙𝐾(𝐾)𝜕𝐱𝐾(𝐾)𝜆𝐾(𝐾)𝑇𝑑𝐱𝐾(𝐾)𝑑𝑡𝑘+𝐿𝑖𝑘𝐱𝑘(𝑘),𝐮𝑘(𝑘),𝐬𝑘(𝑘)𝐿𝑖𝑘+1𝐱𝑘+1(𝑘),𝐮𝑘+1(𝑘),𝐬𝑘+1(𝑘)+𝜆𝑘(𝑘)𝑇𝑑𝐱𝑘(𝑘)𝑑𝑡𝑘𝜆𝑘+1(𝑘)𝑇𝑓𝑖𝑘+1𝐱𝑘+1(𝑘),𝐮𝑘+1(𝑘)𝛼𝑇𝑘𝜕𝑝𝑖𝑘𝜕𝐱𝑘(𝑘)𝑑𝐱𝑘(𝑘)𝑑𝑡𝑘𝐾1𝑗=𝑘𝛼𝑇𝑗𝜕𝑔𝑖𝑗𝜕𝐱𝑗(𝑗)𝑑𝐱𝑗(𝑗)𝑑𝑡𝑘=𝐿𝑖𝑘𝐱𝑘(𝑘),𝐮𝑘(𝑘),𝐬𝑘(𝑘)𝐿𝑖𝑘+1𝐱𝑘+1(𝑘),𝐮𝑘+1(𝑘),𝐬𝑘+1(𝑘)+𝜆𝑘(𝑘)𝑇𝑓𝑖𝑘𝐱𝑘(𝑘),𝐮𝑘(𝑘)𝜆𝑘+1(𝑘)𝑇𝑓𝑖𝑘+1𝐱𝑘+1(𝑘),𝐮𝑘+1(𝑘)𝐾1𝑗=𝑘𝛼𝑇𝑗𝜕𝑔𝑖𝑗𝜕𝐱𝑗(𝑗)𝑑𝐱𝑗(𝑗)𝑑𝑡𝑘𝛼𝑇𝐾𝜕𝜗𝜕𝐱𝐾(𝐾)𝑑𝐱𝐾(𝐾)𝑑𝑡𝑘.(4.26)

Note that when second-order derivatives are needed, there is no difficulty to obtain the second-order derivatives following the above procedure.

4.2. Parametrization Method

To obtain the numerical solution to optimal control for hybrid systems, continuous control profiles are parameterized on each mode active horizon in this section. Then the numerical solution to optimal controls can be computed based on the obtained sensitivity analysis results. The basic idea behind the proposed method using finite parameterizations of the controls is to transcribe the original infinite dimensional problem, that is, 𝐶-problem, into a finite dimensional nonlinear programming problem, that is, 𝑃-problem [25]. Here, the parametrization method that the control profiles are approximated by a family of Lagrange form polynomials is implemented.

Partition each horizon [𝑗1,𝑗) into 𝑁𝑗 elements as 𝑗1=𝜏𝑗0<𝜏𝑗1<<𝜏𝑗𝑁𝑗=𝑗 where 𝜏𝑗𝑙 are referred to as collocation points, 𝑙=0,,𝑁𝑗. Let 𝐮𝑗𝑙 denote the value of 𝐮𝑗 at 𝜏𝑗𝑙, 𝑙=0,,𝑁𝑗. Thus, the control variable 𝐮𝑗 is represented approximately by a Lagrange interpolation profile for 𝑗=1,,𝐾, 𝐮𝑗(𝜏)=𝑁𝑗𝑙=0̂𝑙𝑙(𝜏)𝑢𝑗𝑙[,𝜏𝑗1,𝑗),(4.27) where ̂𝑙𝑙(𝜏)=𝑁𝑗𝑚=0,𝑚𝑙((𝜏𝜏𝑗𝑚)/(𝜏𝑗𝑙𝜏𝑗𝑚)). 𝐬𝑗 is also parameterized by 𝐬𝑗(𝜏)=𝑁𝑗𝑙=0̂𝑙𝑙(𝜏)𝐬𝑗𝑙[,𝜏𝑗1,𝑗),(4.28) where 𝐬𝑗𝑙 is the value of 𝐬𝑗 at the collocation points 𝜏𝑗𝑙, 𝑙=0,,𝑁𝑗.

As a result, based on the obtained derivatives, the numerical solution of 𝑢 and 𝜃 to optimal control for the hybrid systems can be solved simultaneously and efficiently by adopting gradient-based algorithms as described in Xu and Antsaklis [3] and Egerstedt et al. [6]. Note that the derivatives are functions of costate 𝜆𝑗 as formulated in Theorem 4.2. When control polynomial profiles are implemented, a multipoint boundary value problem about state and costate expressed by (3.6), (3.7), and (3.10) will be solved, which produces the derivatives.

Although the Lagrange interpolation profiles may cause the state or/and control trajectories violate their constraints, this parameterizations method has been proved useful in practice. Moreover, there are some techniques to decrease the defect [1, 2].

Remark 4.4. Control variable 𝐮𝑗 can be approximated by several piecewise Lagrange interpolation profiles by further partitioning the element [𝑗1,𝑗). More detail of the parameterizations methods can be found in Vassiliadis et al. [1, 2], Kameswaran and Biegler [26], and the references therein. Only one Lagrange interpolation profile is used here to show the process of the proposed method.

5. Some Examples

To illustrate the effectiveness of the developed method, two examples with different situations are presented in the following. Numerical examples are conducted on an ThinkPad X61 2.10-GHz PC with 2G of RAM. The program is implemented using MatLab 7. The order of Lagrange polynomials in the examples is 3.

Example 5.1. The prototype of this example comes from Vassiliadis et al. [1]. The hybrid system consists of two batch reactors as shown in Figure 1. The first reactor denoted by mode 1 is fitted with a heating coil which can be used to manipulate the reactor temperature 𝑢 over time and is initially loaded with 0.1 m3 of an aqueous solution of component 𝑥1 of concentration 2000 mol/m3. This reacts to form components 𝑥2 according to the consecutive reaction scheme 2𝑥1𝑥2. After completion of the first reaction, an amount of dilute aqueous solution of component 𝑥2 of concentration 600 mol/m3 is added instantaneously to the products of the first reactor, and the mixture is loaded into the second reactor denoted by mode 2 where the reaction 𝑥2𝑥3 takes place under isothermal conditions at a fixed temperature. The decision variables are the temperature 𝑢 of the mode 1, and the durations of the two mode over the horizon [0,180]. The dynamics of the hybrid systems can be described by
Mode 1: ̇𝑥1=0.0888𝑒(2500/𝑢)𝑥21,̇𝑥2=0.0444𝑒(2500/𝑢)𝑥216889.0𝑒(5000/𝑢)𝑥2,̇𝑥3=0.(5.1)
Mode 2: ̇𝑥1=0,̇𝑥2=0.07𝑥28.0×105𝑥22,̇𝑥3=0.02𝑥2,(5.2) with 𝑥(0)=[200000]𝑇. The system transits once at 𝑡=𝑡1(𝑡0<𝑡1<𝑡𝑓) from mode 1 to 2 with 𝑥1(𝑡1)=𝑥1(𝑡1)/1.7,𝑥2(𝑡1)=(𝑥2(𝑡1)+420)/1.7. The OCPHS is to find an optimal mode transition instant 𝑡1 and an optimal input 298𝑢(𝑡)398, 𝑡[𝑡0,𝑡1], to maximize the cost functional max𝑡1,𝑢𝑥3𝑡𝑓,(5.3) with 𝑥3(𝑡𝑓)150 must be satisfied.

581040.fig.001
Figure 1: Two batch reactors system.

By using the proposed method, the optimal mode transition instant is 𝑡1=105 and the corresponding optimal cost is 𝐽=150.0285. The corresponding continuous control and state trajectories are shown in Figure 2. In Vassiliadis et al. [1], the transition instants and the optimal cost are 𝑡1=106, 𝐽=150.294, respectively, which are solved by software package DAEOPT.

581040.fig.002
Figure 2: State trajectories and control input of Example 5.1.

Example 5.2. Example 5.2 comes from Xu and Antsaklis [3] and is also reconsidered by Hwang et al. [9]. Different from the example in the two references, the control constraint is imposed. The example can be referred to as autonomous switching hybrid systems with mode invariants. Consider the hybrid system consisting of
Mode 1: 11̇𝑥=1.5001𝑥+𝑢,(5.4)
Mode 2: 11̇𝑥=0.50.8660.8660.5𝑥+𝑢,(5.5) with 𝑥0=[11]𝑇. Assume that 𝑡0=0, 𝑡𝑓=2 and the system transits once at 𝑡=𝑡1(𝑡0<𝑡1<𝑡𝑓) from Mode 1 to 2 when the state trajectories intersect the linear manifold defined by 𝑚(𝑥)=𝑥1+𝑥27=0. Mode 1 is active with its mode invariant 𝑥1+𝑥27<0 and Mode 2 is active with its mode invariant 𝑥1+𝑥27>0. The OCPHS is to find an optimal mode transition instant 𝑡1 and an optimal input 𝑢(𝑡)[1,1] such that the cost functional 𝐽𝑡1=1,𝑢2𝑥1𝑡𝑓102+𝑥2𝑡𝑓62+𝑡𝑓𝑡0𝑢2(𝑡)𝑑𝑡(5.6) is minimized.

By using the method developed here, the optimal mode transition instant is 𝑡1=1.1857 and the corresponding optimal cost is 𝐽=0.1246. The corresponding continuous control and state trajectories are shown in Figure 3. In Xu and Antsaklis [3], the transition instants and the optimal cost are 𝑡1=1.1624, 𝐽=0.1130, respectively. The bad performance results from that the optimal control is approximated by polynomial.

581040.fig.003
Figure 3: State trajectories and control input of Example 5.2.

6. Conclusions

The optimal control problem for hybrid systems (OCPHS) with mode invariants and control constraints is addressed under a priori fixed mode transition order. By introducing new independent variables and auxiliary algebraic variables, the original OCPHS is transformed into an equivalent optimal control problem, and the optimality conditions for the OCPHS is stated. Based on the optimality conditions, the derivatives of the objective functional w.r.t control variables, that is, mode transition instant sequence and admissible continuous control functions, are established analytically. As a result, a control vector parametrization method is implemented to obtain the numerical solution by using gradient-based algorithms with the obtained derivatives. Compared with the existing results on the OCPHS with fixed mode transition order, the settings cover not only the control constraints but also the continuous states constraints, which makes the obtained results more general. Note that when no information about the mode transition sequence is known a priori, the discrete model methods formulated in Bemporad and Morari [27], Barton et al. [15], and Song et al. [28] seem appropriate. In addition, when uncertainties are considered in the systems, the reader is referred to Hu et al. [29] and the references therein.

Appendix

For any 𝜏[𝑘1,𝑘),𝑘=1,,𝐾, let 𝐮𝑘(𝜏)𝐔𝑖𝑘 be given and let 𝛿𝐮𝑘(𝜏)𝐔𝑖𝑘 be arbitrary but fixed. Define a perturbation of 𝐮𝑘 as 𝐮𝑘(𝜏;𝜀)=𝐮𝑘(𝜏)+𝜀𝛿𝐮𝑘(𝜏),(A.1) where 𝜀 is arbitrarily small such that 𝐮𝑘(𝜏;𝜀)𝐔𝑖𝑘. For the time being, assume that the other controls, 𝐮𝑗,𝑗=1,,𝐾,𝑗𝑘, be given and fixed. For brevity, let 𝐱𝑗 and 𝐱𝑗(;𝜀) denote the state trajectories corresponding to 𝐮𝑘 and 𝐮𝑘(𝜏;𝜀), respectively. Similarly, let 𝜆𝑗 and 𝜆𝑗(;𝜀) denote the costate trajectories corresponding to 𝐮𝑘 and 𝐮𝑘(𝜀), respectively, which are the solutions of the costate equations 𝐱𝑗(;𝜀)=𝐱𝑗()+𝜀𝛿𝐱𝑗𝜆(),𝑗(;𝜀)=𝜆𝑗()+𝜀𝛿𝜆𝑗().(A.2)

Proof of (4.2) in Lemma 4.1. When 𝑗=1,,𝑘1, obviously in these cases 𝐱𝑗 is independent of 𝐮𝑘, that is, 𝛿𝐱𝑗(𝑗;𝜀)=0, which leads to 𝛿𝐱𝑗(𝑗)𝛿𝐮𝑘=0,𝑗=1,,𝑘1.(A.3)
Case i (𝑗=𝑘). Since 𝛿̇𝐱𝑘=𝑡𝑘𝑡𝑘1𝜕𝑓𝑖𝑘𝜕𝐱𝑘𝛿𝐱𝑘+𝜕𝑓𝑖𝑘𝜕𝐮𝑘𝛿𝐮𝑘,(A.4) with 𝛿𝐱𝑘(𝑘1)=0, thus we have 𝛿𝐱𝑘(𝑘)=𝑘𝑘1Φ𝑘𝑡(𝑘,𝜏)𝑘𝑡𝑘1𝜕𝑓𝑖𝑘𝜕𝐮𝑘𝛿𝐮𝑘𝑑𝜏,(A.5) where Φ𝑘 is the state transition matrix defined in Section 3. Based on the definition of functional derivative, there exists 𝛿𝐱𝑘(𝑘)𝛿𝐮𝑘=𝑡𝑘𝑡𝑘1Φ𝑘(𝑘,𝜏)𝜕𝑓𝑖𝑘𝜕𝐮𝑘Γ𝑘(𝜏).(A.6)
Case (ii) (𝑗=𝑘+1,,𝐾). In this case, 𝛿̇𝐱𝑗𝑡(𝜏;𝜀)=𝑗𝑡𝑗1𝜕𝑓𝑖𝑗𝜕𝐱𝑗𝛿𝐱𝑗[,𝜏𝑗1,𝑗),(A.7) which gives rise to 𝛿𝐱𝑗(𝑗;𝜀)=Φ𝑗(𝑗,𝑗1)𝛿𝐱𝑗(𝑗1).(A.8)
At mode transition instant 𝑡𝑗,𝑗=1,,𝐾1, 𝐱𝑗+1(𝑗)=𝜓𝑖𝑗+1(𝐱𝑗(𝑗)) holds, which results in 𝛿𝐱𝑗+1(𝑗)=𝑑𝜓𝑖𝑗+1𝑑𝐱𝑗(𝑗)𝛿𝐱𝑗(𝑗).(A.9)
Substituting (A.9) into (A.8), we obtain 𝛿𝐱𝑗(𝑗)=𝑗𝑙=𝑘+1Φ𝑙(𝑙,𝑙1)𝑑𝜓𝑖𝑙𝑑𝐱𝑙1((𝑙1))𝛿𝐱𝑘(𝑘).(A.10)
According to the definition of functional derivative, we have 𝛿𝐱𝑗(𝑗)𝛿𝐮𝑘=𝑗𝑙=𝑘+1Φ𝑙(𝑙,𝑙1)𝑑𝜓𝑖𝑙𝑑𝐱𝑙1((𝑙1))Γ𝑘(𝜏).(A.11)
This completes the proof.

Before proving the 𝛿𝐽/𝛿𝐮𝑘 in Theorem 4.2, Lemma A.1 is firstly given as follows.

Lemma A.1. For any 𝑗=𝑘+1,,𝐾, 𝛿𝑗𝑗1𝐿𝑖𝑗𝐱𝑗,𝐮𝑗,𝐬𝑖𝑑𝜏=𝜆𝑗(𝑗1)𝑇𝛿𝐱𝑗(𝑗1)𝜆𝑗(𝑗)𝑇𝛿𝐱𝑗(𝑗).(A.12)

Proof. Note that 𝛿𝑗𝑗1𝐿𝑖𝑗𝐱𝑗,𝐮𝑗,𝐬𝑗𝑑𝜏=𝛿𝑗𝑗1𝐻𝑗𝜆𝑗,𝐱𝑗,𝐮𝑗,𝐬𝑗𝜆𝑇𝑗𝑓𝑖𝑗=𝑑𝜏𝑗𝑗1𝜕𝐻𝑗𝜕𝐱𝑗𝛿𝐱𝑗+𝜕𝐻𝑗𝜕𝜆𝑗𝛿𝜆𝑗𝛿𝜆𝑗𝑇𝑓𝑖𝑗𝜆𝑇𝑗𝛿𝑓𝑖𝑗𝑑𝜏.(A.13)
Since the following holds by Theorem 3.2: 𝜕𝐻𝑗𝜕𝐱𝑗𝑇̇𝜆=𝑗,𝜕𝐻𝑗𝜕𝜆𝑗𝑇=𝑓𝑖𝑗,(A.14) therefore, 𝛿𝑗𝑗1𝐿𝑖𝑗𝐱𝑗,𝐮𝑗,𝐬𝑗𝑑𝜏=𝑗𝑗1̇𝜆𝑗𝑇𝛿𝐱𝑗+𝜆𝑇𝑗𝛿𝑓𝑖𝑗𝑑𝜏=𝑗𝑗1̇𝜆𝑗𝑇𝛿𝐱𝑗+𝜆𝑇𝑗𝛿̇𝐱𝑖𝑗𝑑𝜏=𝑗𝑗1𝑑𝜆𝑑𝜏𝑇𝑗𝛿𝐱𝑗𝑑𝜏=𝜆𝑗(𝑗1)𝑇𝛿𝐱𝑗(𝑗1)𝜆𝑗(𝑗)𝑇𝛿𝐱𝑗(𝑗).(A.15)

Obviously, when 𝑗=𝑘, we have 𝛿𝑘𝑘1𝐿𝑖𝑘𝐱𝑘,𝐮𝑘,𝐬𝑘𝑑𝜏=𝜆𝑘(𝑘1)𝑇𝛿𝐱𝑘(𝑘1)𝜆𝑘(𝑘)𝑇𝛿𝐱𝑘(𝑘)+𝑘𝑘1𝜕𝐻𝑘𝜕𝐮𝑘𝛿𝐮𝑘𝑑𝜏.(A.16)

Proof of 𝛿𝐽/𝛿𝐮𝑘 in Theorem 4.2. 𝐽(𝜃,𝐮(𝜀),𝐬) can be rewritten by 𝐱𝐽(𝜃,𝐮(𝜀),𝐬)=𝜙𝐾+(𝐾)𝑘1𝑗=1𝑗𝑗1𝐿𝑖𝑗𝐱𝑗,𝐮𝑗,𝐬𝑗𝑑𝜏+𝑘𝑘1𝐿𝑖𝑘𝐱𝑘(𝜀),𝐮𝑘(𝜀),𝐬𝑘+𝑑𝜏𝐾𝑗=𝑘+1𝑗𝑗1𝐿𝑖𝑗𝐱𝑗(𝜀),𝐮𝑗,𝐬𝑗𝑑𝜏+𝐾1𝑗=1𝜑𝑖𝑗𝑖𝑗+1𝐱𝑗(𝑗).(A.17) Applying a 𝛿-operation to (A.17) leads to 𝛿𝑑𝐽=𝐽(𝜌,𝐮(𝜀),𝐬)||||𝑑𝜀𝜀=0=𝐱𝜕𝜙𝐾(𝐾)𝜕𝐱𝐾(𝐾)𝛿𝐱𝐾(𝐾)+𝑘𝑘1𝜕𝐻𝑘𝜕𝐮𝑘𝛿𝐮𝑘+𝑑𝜏𝐾𝑗=𝑘𝜆𝑗(𝑗1)𝑇𝛿𝐱𝑗(𝑗1)𝜆𝑗(𝑗)𝑇𝛿𝐱𝑗(𝑗)+𝐾1𝑗=1𝜕𝜑𝑖𝑗𝑖𝑗+1𝜕𝐱𝑗(𝑗)𝛿𝐱𝑗(𝑗)=𝐱𝜕𝜙𝐾(𝐾)𝜕𝐱𝐾(𝐾)𝛿𝐱𝐾(𝐾)+𝑘𝑘1𝜕𝐻𝑘𝜕𝐮𝑘𝛿𝐮𝑘𝑑𝜏+𝜆𝑘(𝑘1)𝑇𝛿𝐱𝑘(𝑘1)𝐾1𝑗=𝑘𝜆𝑗(𝑗)𝑇𝛿𝐱𝑗(𝑗)𝜆𝑗+1(𝑗)𝑇𝛿𝐱𝑗+1(𝑗)𝜕𝜑𝑖𝑗𝑖𝑗+1𝜕𝐱𝑗(𝑗)𝛿𝐱𝑗(𝑗)𝜆𝐾(𝐾)𝑇𝛿𝐱𝐾(𝐾).(A.18)
Due to Theorem 3.2 and (A.9), 𝛿𝐽 can be reformulated by 𝛿𝐱𝐽=𝜕𝜙𝐾(𝐾)𝜕𝐱𝐾(𝐾)𝜆𝐾(𝐾)𝑇𝛿𝐱𝐾(𝐾)+𝑘𝑘1𝜕𝐻𝑘𝜕𝐮𝑘𝛿𝐮𝑘𝑑𝜏𝐾1𝑗=𝑘𝛼𝑇𝑗𝜕𝑔𝑖𝑗𝜕𝐱𝑗(𝑗)𝛿𝐱𝑗(𝑗)=𝑘𝑘1𝜕𝐻𝑘𝜕𝐮𝑘𝛿𝐮𝑘𝑑𝜏𝐾1𝑗=𝑘𝛼𝑇𝑗𝜕𝑔𝑖𝑗𝜕𝐱𝑗(𝑗)𝛿𝐱𝑗(𝑗)𝛼𝑇𝐾𝜕𝜗𝜕𝐱𝐾(𝐾)𝛿𝐱𝐾(𝐾).(A.19)
Then according to the definition of functional derivative, we have 𝛿𝐽𝛿𝐮𝑘=𝜕𝐻𝑘𝜕𝐮𝑘𝐾1𝑗=𝑘𝛼𝑇𝑗𝜕𝑔𝑖𝑗𝜕𝐱𝑗(𝑗)𝛿𝐱𝑗(𝑗)𝛿𝐮𝑘𝛼𝑇𝐾𝜕𝜗𝜕𝐱𝐾(𝐾)𝛿𝐱𝐾(𝐾)𝛿𝐮𝑘.(A.20)
Obviously, the functional derivative of 𝐽 with respect to 𝐬𝑘 can be directly given by 𝛿𝐽𝛿𝐬𝑘=𝜕𝐻𝑘𝜕𝐬𝑘.(A.21)
This completes the proof.

Acknowledgments

The authors would like to express their sincere thanks to Professor Michael Fu at the University of Maryland (College Park) for suggestions on this work. This work was supported by the NSF under grant 60974023 of China, the State Key Development Program for Basic Research of China (2012CB720503), and the Fundamental Research Funds for the Central Universities.

References

  1. V. S. Vassiliadis, R. W. H. Sargent, and C. C. Pantelides, “Solution of a class of multistage dynamic optimization problems. 1. Problems without path constraints,” Industrial and Engineering Chemistry Research, vol. 33, no. 9, pp. 2111–2122, 1994. View at Publisher · View at Google Scholar · View at Scopus
  2. V. S. Vassiliadis, R. W. H. Sargent, and C. C. Pantelides, “Solution of a class of multistage dynamic optimization problems. 2. Problems with path constraints,” Industrial and Engineering Chemistry Research, vol. 33, no. 9, pp. 2123–2133, 1994. View at Publisher · View at Google Scholar · View at Scopus
  3. X. Xu and P. J. Antsaklis, “Optimal control of switched systems based on parameterization of the switching instants,” IEEE Transactions on Automatic Control, vol. 49, no. 1, pp. 2–16, 2004. View at Publisher · View at Google Scholar
  4. P. J. Antsaklis and A. Nerode, “Special issue on hybrid control systems,” IEEE Transactions on Automatic Control, vol. 43, no. 4, 1998.
  5. P. E. Caines and Y.-J. Wei, “Hierarchical hybrid control systems: a lattice-theoretic formulation,” IEEE Transactions on Automatic Control, vol. 43, no. 4, pp. 501–508, 1998. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  6. M. Egerstedt, Y. Wardi, and H. Axelsson, “Transition-time optimization for switched-mode dynamical systems,” IEEE Transactions on Automatic Control, vol. 51, no. 1, pp. 110–115, 2006. View at Publisher · View at Google Scholar
  7. D. Flieller, P. Riedinger, and J. P. Louis, “Computation and stability of limit cycles in hybrid systems,” Nonlinear Analysis, vol. 64, no. 2, pp. 352–367, 2006. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  8. C. G. Cassandras and J. Lygeros, Stochastic Hybrid Systems, chapter 1, 2, 6, 7, Taylor & Francis, Boca Raton, Fla, USA, 2007.
  9. I. Hwang, J. Li, and D. Du, “A numerical algorithm for optimal control of a class of hybrid systems: differential transformation based approach,” International Journal of Control, vol. 81, no. 2, pp. 277–293, 2008. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  10. H. J. Sussmann, “A maximum principle for hybrid optimal control problems,” in Proceedings of the 38th IEEE Conference on Decision and Control (CDC '99), pp. 425–430, Phoenix, Ariz, USA, December 1999. View at Scopus
  11. M. S. Shaikh and P. E. Caines, “On the hybrid optimal control problem: theory and algorithms,” IEEE Transactions on Automatic Control, vol. 52, no. 9, pp. 1587–1603, 2007. View at Publisher · View at Google Scholar
  12. A. V. Dmitruk and A. M. Kaganovich, “The hybrid maximum principle is a consequence of Pontryagin maximum principle,” Systems & Control Letters, vol. 57, no. 11, pp. 964–970, 2008. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  13. A. Bensoussan and J. L. Menaldi, “Stochastic hybrid control,” Journal of Mathematical Analysis and Applications, vol. 249, no. 1, pp. 261–288, 2000. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  14. M. S. Branicky, V. S. Borkar, and S. K. Mitter, “A unified framework for hybrid control: model and optimal control theory,” IEEE Transactions on Automatic Control, vol. 43, no. 1, pp. 31–45, 1998. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  15. P. I. Barton, C. K. Lee, and M. Yunt, “Optimization of hybrid systems,” Computers and Chemical Engineering, vol. 30, no. 10–12, pp. 1576–1589, 2006. View at Publisher · View at Google Scholar · View at Scopus
  16. C. Seatzu, D. Corona, A. Giua, and A. Bemporad, “Optimal control of continuous-time switched affine systems,” IEEE Transactions on Automatic Control, vol. 51, no. 5, pp. 726–741, 2006. View at Publisher · View at Google Scholar
  17. H. Axelsson, Y. Wardi, M. Egerstedt, and E. I. Verriest, “Gradient descent approach to optimal mode scheduling in hybrid dynamical systems,” Journal of Optimization Theory and Applications, vol. 136, no. 2, pp. 167–186, 2008. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  18. E. R. Johnson and T. D. Murphey, “Second order switching time optimization for time-varying nonlinear systems,” in Proceedings of the 48th IEEE Conference on Decision and Control (CDC '09), pp. 5281–5286, Shanghai, China, 2009. View at Publisher · View at Google Scholar · View at Scopus
  19. S. A. Attia, V. Azhmyakov, and J. Raisch, “On an optimization problem for a class of impulsive hybrid systems,” Discrete Event Dynamic Systems, vol. 20, no. 2, pp. 215–231, 2010. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  20. F. Taringoo and P. E. Caines, “Gradient geodesic and Newton geodesic HMP algorithms for the optimization of hybrid systems,” Annual Reviews in Control, vol. 35, pp. 187–198, 2011.
  21. X. Xu and P. J. Antsaklis, “A dynamic programming approach for optimal control of switched systems,” in Proceedings of the 39th IEEE Confernce on Decision and Control (CDC '00), pp. 1822–1827, Sydney, Australia, December 2000. View at Scopus
  22. K. Gokbayrak and C. G. Cassandras, “Hybrid controllers for hierarchically decomposed systems,” in Proceedings of the Hybrid System Control Confernce, pp. 117–129, March 2000. View at Scopus
  23. L. S. Jennings, K. L. Teo, V. Rehbock, and W. X. Zheng, “Optimal control of singular systems with a cost on changing control,” Dynamics and Control, vol. 6, no. 1, pp. 63–89, 1996. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  24. C. Song and P. Li, “Near optimal control for a class of stochastic hybrid systems,” Automatica, vol. 46, no. 9, pp. 1553–1557, 2010. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  25. R. Findeisen, L. Imsland, F. Allgöwer, and B. A. Foss, “State and output feedback nonlinear model predictive control: an overview,” European Journal of Control, vol. 9, no. 2-3, pp. 190–206, 2003. View at Publisher · View at Google Scholar · View at Scopus
  26. S. Kameswaran and L. T. Biegler, “Simultaneous dynamic optimization strategies: recent advances and challenges,” Computers and Chemical Engineering, vol. 30, no. 10–12, pp. 1560–1575, 2006. View at Publisher · View at Google Scholar · View at Scopus
  27. A. Bemporad and M. Morari, “Control of systems integrating logic, dynamics, and constraints,” Automatica, vol. 35, no. 3, pp. 407–427, 1999. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  28. C. Song, B. Wu, and P. Li, “A hybrid model-based optimal control method for nonlinear systems using simultaneous dynamic optimization strategies,” Journal of Process Control, vol. 22, no. 5, pp. 852–860, 2012.
  29. J. Hu, Z. Wang, H. Gao, and L. K. Stergioulas, “Probability-guaranteed H finite-horizon filtering for a class of nonlinear time-varying systems with sensor saturation,” System and Control Letters, vol. 61, no. 4, pp. 477–484, 2012.