Shock and Vibration

Volume 2016, Article ID 9641075, 9 pages

http://dx.doi.org/10.1155/2016/9641075

## Optimal Vibration Control of a Class of Nonlinear Stochastic Systems with Markovian Jump

Department of Mechanics, State Key Laboratory of Fluid Power Transmission and Control, Zhejiang University, Hangzhou 310027, China

Received 15 May 2015; Accepted 26 August 2015

Academic Editor: Mickaël Lallart

Copyright © 2016 R. H. Huan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

The semi-infinite time optimal control for a class of stochastically excited Markovian jump nonlinear system is investigated. Using stochastic averaging, each form of the system is reduced to a one-dimensional partially averaged Itô equation of total energy. A finite set of coupled dynamical programming equations is then set up based on the stochastic dynamical programming principle and Markovian jump rules, from which the optimal control force is obtained. The stationary response of the optimally controlled system is predicted by solving the Fokker-Planck-Kolmogorov (FPK) equation associated with the fully averaged Itô equation. Two examples are worked out in detail to illustrate the application and effectiveness of the proposed control strategy.

#### 1. Introduction

Markovian jump system (MJS) is a class of hybrid systems, which has different operation modes or forms governed by a Markov process. This class of systems can be used to represent complex real systems, which may experience abrupt variation in their structures and parameters. It is well known that random form switching can give rise to unexpected phenomena of the whole MJSs, even if the dynamics in each form is simple and well understood. Development of methodology for analysis of such systems is thus much deserving.

MJSs have been studied with growing interest and activity in recent years since they are first introduced by Kats and Krasovskii [1]. For dynamics analysis, several important criteria for stochastic stability of MJSs have been established [2, 3]. The stationary response of the stochastically excited nonlinear MJSs is studied by Huan et al. [4]. For the problem of optimal control, Krasosvkii and Lidskii studied the LQR control of Markovian jump linear systems firstly [5]. As a consequence, considerable attention has been paid to the optimal control problem of MJSs [6, 7]. Sworder solved the jump linear Gaussian problem for finite time horizon [8]. The infinite time horizon problem of linear MJSs was studied by Wonham [9] and Fragoso and Hemerly [10]. The optimal control solution for linear MJSs subject to Gaussian input and measurement noise is given by Ji and Chizeck [11]. Ghosh et al. investigated the ergodic control problem of switching diffusion [12]. However, most of the published results are applicable to linear MJSs. Far less is known about the optimal control of nonlinear MJSs, particularly for stochastically excited nonlinear MJSs.

Recently, a nonlinear optimal control strategy of stochastic excited nonlinear systems has been proposed by Zhu based on the stochastic averaging method [13] and stochastic dynamical programming principle. It is proved that this strategy is quite effective for response alleviation of nonlinear stochastic systems. The goal of this paper is to extend the strategy for nonlinear stochastic system to the one for nonlinear stochastic MJSs.

An infinite time optimal control problem for single degree-of-freedom (SDOF), stochastically excited, nonlinear, Markovian jump system is investigated in this paper. The organization of this paper is as follows. In Section 2, the equation of SDOF nonlinear Markovian jump system is examined. Stochastic averaging method is applied to this system in Section 3. The partially averaged equation of total energy for each form of the original system is obtained. A finite set of coupled dynamical programming equations is then set up in Section 4, from which the optimal control force is determined. The Fokker-Planck-Kolmogorov (FPK) equation associated with the fully averaged Itô equation is also derived and solved. In Section 5, two examples are worked out in detail to illustrate the application and effectiveness of the proposed control strategy.

#### 2. Formulation of Control Problem

Consider a SDOF controlled, stochastically driven nonlinear system with Markovian jump governed bywhere is nonlinear stiffness; is a small parameter; denotes light damping with Markovian jump; is the feedback control force; represent the Markovian jump coefficients of weakly external and (or) parametric stochastic excitations; are independent Gaussian white noises with zero means and intensities .

The parameter is a continuous-time Markovian jump process with finite discrete state-space and transition matrix , where are real numbers such that, for , , and for all , . The transition probability is given by [14, 15]where is such that .

Our primary goal here is to design an optimal controller which will result in the minimum of a specified performance. For infinite-horizon control problem, the performance index is specified bywhere denotes an expectation operation; is the terminal time of control; is cost function. If random excitations and Markovian jump process are ergodic, then the response of controlled system approaches to stationary and ergodic as . In this case, the performance index (3) is rewritten as

Equations (1) and (4) constitute the mathematical formulation of the infinite-horizon optimal control problem for a SDOF nonlinear system undergoing Markovian jump. In this paper, we consider the perfect observation case, which means that the system state () and values of are all available at each time .

#### 3. Partially Averaged Equation

Let and ; (1) can be converted to the following Itô stochastic differential equations [11, 16]:where are independent standard Wiener processes; and are the generalized displacement and generalized momentum, respectively; . The Hamiltonian function (total energy) of the Hamiltonian system associated with system (5) isSuppose that is a slow jump process and independent of system’s state. That means that, for original system (1) or (5), the system jumps among the forms or modes slowly and independently. We consider one of the forms by letting the Markovian jump process be arbitrarily fixed at . When , denote , , and . Since is a function of and , the Itô equation for can be derived from (5) by using Itô differential rule as follows [17]:

In the case of light damping and weak excitations, the Hamiltonian in system (7) is a slowly varying process while the generalized displacement in (5) is a rapidly varying process. Since the slowly varying process is essential for describing the long-term behavior of the system, the stochastic averaging method is used to average out the rapidly varying process and to yield the following partially averaged Itô equation for [13, 18, 19]:where The region of integration is , and the parameterNote that the second term on the right-hand of (8) is unknown since are unknown function of , , and at this stage.

Equation (8) is only valid when . As Markovian jumps are allowed so that takes values from , (8) can be extended so thatwhere , , and change as jumps, so that , , and .

To be consistent with the partially averaged equation (11), performance index (4) is also partially averaged: that is,The original optimal control problem (1) and (4) has been converted to the partially averaged optimal control problem (11) and (12). The partially averaged optimal control problem is much simpler than the original one since (11) is only one-dimensional. It has been proved [20] that the optimal control for the partially averaged control problem is quasi-optimal control for the original one.

#### 4. Optimal Control Law

For the proposed control problem, we introduce the value function:

According to the stochastic dynamical programming principle and Markovian jump rules, the following dynamical programming equation can be established [16]:where are the transition rates described in (2). Equation (14) is a finite set of second-order partial differential equations indexed by . These equations are coupled through the zero-order terms .

The necessary condition for minimizing the left-hand side of (14) is

Let the cost function be of the formwhere , and is a positive constant. By substituting (16) into (15) and exchanging the order of the derivative with respect to and averaging in (15), the expression of the optimal control force is determined:Note that is generally a nonlinear functional of . Hence, is a nonlinear function of and ; that is, the optimal feedback control is nonlinear. To ensure that the control force is indeed optimal, the following sufficient condition should be satisfied:

Substituting the expression for in (17) into (14) yields the final dynamical programming equation:where

The value function can be obtained by solving (19). The optimal control force is then determined by substituting into expression (17). By inserting the optimal control force into (11) and averaging the term , the following completely averaged Itô equation is obtained:The FPK equation associated with the fully averaged Itô equation (21) is [4, 21, 22]where is the probability density function of total energy for optimally controlled system (1) while the system operating in th form ( for uncontrolled system can be obtained by letting ). The initial condition isand boundary conditions are

The FPK equation (22) is a finite set of second-order partially differential equations, which are coupled through the zero-order terms. Equation (22) does not admit easy solution, analytically or numerically. However, in practical application we are more interested in the stationary solution of FPK equation (22). In this case, FPK equation (22) is reduced by letting . Then, the stationary probability density is obtained readily from solving (22) numerically. The stationary probability density can be obtained from as follows:where is a normalization constant. The stationary probability density of the generalized displacement and the mean value of the total energy are then obtained aswhereis the stationary joint probability density of the generalized displacement and generalized momentum.

#### 5. Numerical Example

To demonstrate the application and to assess the effectiveness of the proposed control strategy, numerical results are obtained for two examples.

##### 5.1. Example 1

Consider a controlled van der Pol-Rayleigh oscillator subjected to both external and parametric random excitations, which is capable of independent Markovian jump and governed bywhere , , and are Markovian jump coefficients of linear and quasi-linear damping, respectively; and denote Markovian jump coefficients of external and parametric random excitations, respectively; is the feedback control force; are independent Gaussian white noises with zero mean and intensities . is a finite-state continuous-time Markov process pointing to the system’s form with the transition probability defined in (2).

We consider the 3-form case here that means and . Prescribe the transition rate between the form and the form by a transition matrix .

Let and . The Hamiltonian associated with system (28) is

As a Hamiltonian system, (28) can be expressed as

Upon stochastic averaging, (11) for system (28) is obtained, for which , and

For the proposed control strategy, the partially averaged cost function is of the form of (16) with

Following (17), the expression of the optimal control force is

Substituting in (33) into the dynamical programming equation (14), completing the averaging yields the following final coupled dynamical programming equations:

To solve (34), the value function is supposed to be of the following form:

Since the optimal control force is function of , only the coefficients and are determined by inserting (32) and (35) into (34):

By substituting (35) into (17) with the coefficients determined by (36), the optimal control force is determined. Then, the following completely averaged FPK equation is obtained:where

Solving (37) numerically with , the stationary probability density of optimal controlled system can be obtained. The stationary probability densities of the total energy, of the generalized displacement, and the mean value of the total energy are then obtained using (25) and (26).

Some numerical results are obtained as shown in Figures 1–4 for system parameters ; ; ; ; ; and