Mathematical Problems in Engineering

Volume 2015 (2015), Article ID 808903, 10 pages

http://dx.doi.org/10.1155/2015/808903

## Optimal Control for a Linear System Subject to a General ARIMA Disturbance

^{1}School of Economics, Southwestern University of Finance and Economics, Chengdu, Sichuan 611130, China^{2}School of Finance, Southwestern University of Finance and Economics, Chengdu, Sichuan 611130, China

Received 28 November 2014; Revised 24 January 2015; Accepted 16 February 2015

Academic Editor: Alain Vande Wouwer

Copyright © 2015 Hongyan Xie and Fangyi He. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

A novel run-to-run (R2R) control algorithm based on Kalman filter approach is proposed to deal with a linear system with a general ARIMA() process, in the presence of measurement error and adjustment error together with a random initial bias. We mathematically prove its optimality. The performance of the newly proposed controller and the proportional-integral () controller is evaluated and compared under multiple scenarios through Monte Carlo simulations. Almost all the results reflect the new controller’s superiority.

#### 1. Introduction and Motivation

Optimizing a sequence of actions to obtain some future goal is the general topic of control theory (see [1]). Many optimal control methods have been widely used in actuarial science (e.g., [2, 3]), production management (e.g., [4–6]), and quality management (e.g., [7–9]). Specifically, disturbance rejection is one of the major concerns in quality control problems, such as machine setup adjustment problems and R2R process control problems in semiconductor manufacturing (see [10, 11]). Grubbs [12] originally studies the machine setup adjustment problem and develops an optimal adjustment rule for a linear system with normal disturbances, which is referred to by Trietsch [13] as the “harmonic rule.” Vander Wiel et al. [14] give an optimal control algorithm for a linear system with ARMA() disturbances. For a linear system with IMA() disturbances, Ingolfsson and Sachs [15] and Box et al. [16] introduce the exponential weighted moving average (EWMA) control algorithm and prove its optimality; He et al. [17] recently develop an optimal control algorithm, the ARMA controller, for a linear system with a general ARMA() disturbance.

In this paper, we will extend the results in [17] and develop an optimal control algorithm for a linear system with a general ARIMA() disturbance, where . The ARIMA disturbance has been widely used to describe process dynamics (see [18–21]). Since ARIMA() processes can be used to model a large class of nonstationary disturbances, many machine setup problems and R2R control problems can be solved under this new framework. Similar to [17], the newly proposed controller will consider a more realistic case such that both measurement error and adjustment error exist, and the initial bias of the process is a random variable. We present the problem as follows.

Supposing a process to be controlled can be expressed as where is the state of at time ; is an ARIMA() process satisfying where , , and is a white noise process with mean 0 and variance , that is, ; is the measurement error for and . is the backward shift operator defined by Neither nor can be measured or observed directly. At time , suppose we need to make an adjustment of magnitude to to bring the process output to target in the next run. That is, where is the adjustment error and . In practice, the process adjustment is assumed to be done by adjusting one controllable factor, , via the following model: where is called the process gain and . Assume , . The initial value is assumed to be a random variable with , , and . Without loss of generality, the target is assumed to be 0 in the rest of this paper. For a positive time index , we hope to determine the optimal , that satisfy where is the expectation operator conditional on all the information until time . Problem (6) is a finite-horizon problem. When , we get the infinite-horizon problem that searches the optimal satisfying The purpose of this paper is to find the solutions to both problems (6) and (7).

The rest of this paper is organized as follows. In Section 2 we derive the state-space representation of the system and get the recursive estimation formulae of the system states using Kalman filter. In Section 3, we develop a control algorithm for the system and prove its optimality without normal distribution assumptions. We further give the implementation steps of the controller in practice. Simulation studies are done in Section 4 under multiple scenarios. Section 5 gives an illustrative example for the application of the control algorithm. Concluding remarks are included in Section 6.

#### 2. The State-Space Representation

Let and be the coefficients of in the power series expansion of , . Brockwell and Davis [22] gave a state-space representation for a general ARIMA() process. By extending their results we can get the following Theorem 1 easily.

Theorem 1. *The linear system (1)–(5) has a state-space representation as follows:**where and are the and matrices defined by if and
**
if ; and are the and matrices defined by , if and
**
if ; , , , .*

A proof of Theorem 1 is presented in Appendix A. For simplicity, in the rest of this paper we denote where is defined to be a vector of ones.

That the problems of optimal control and state estimation can be decoupled in certain cases is one of the most fundamental principles in feedback control theory. This is known as the* separation principle* (see [23]). The Kalman filter produces the statistically optimal estimate of the state of system (8)-(9). Note the fact that the disturbances of (8) and (9) are correlated to each other, so we should use the Kalman filter formulae for correlated measurement and process noise. At any time , let us define and and assume and . Using the results on page 123 in Lewis’ book [24], we can directly get the following Lemma 2.

Lemma 2. *For the systems (8)-(9), we have the following recursive formulae:
**
where is called the modified Kalman Gain that
*

*3. The Optimal Control Algorithm*

*In this section, we will derive an optimal control algorithm without the normal distribution assumptions on , , and , . This optimal control algorithm can be applied to both the finite-horizon and infinite-horizon problems.*

*Theorem 3. The optimal control algorithm that solves both
for the system (1)–(5) is
where is updated according to
and (14), (15) are with initial values and .*

*Proof. *For a given positive time index , let and . We can get the Bellman equation as
From (8), it is evident that . Using the fact that
and the property of trace operator, we obtain that
From (13) and the fact that , we have
Let . It is evident that is irrelated to since is updated based on (14) and (15). Now (19) changes to
When , we get
Solving the first order condition, we get that
Putting (25) into (24), we get . When , we have
Solving the first order condition, we get
and . Repeating the procedures above, we can prove that (17) is the optimal adjustment strategy at time and get
At last, we can derive (18) by putting (17) into (13). Since (17) is irrelated to the given time index , the control strategy also solves the infinite-horizon problem. Then we finish the proof.

*In practice, the steps to implement the proposed control algorithm are as follows.*

*Step 0.* Setup the algorithm’s parameters and initial values, such as , , , , , , based on experience or history data.

*Step 1.* Compute based on (15).

*Step 2.* Collect the new observation and compute the adjustment based on (17).

*Step 3.* Update based on (18).

*Step 4.* Update based on (14); let and go back to Step 1.

*Note that Step 0 is an off-line procedure and Step 1 to Step 4 are on-line procedures. As the newly proposed control algorithm is specially designed for adjusting any ARIMA disturbances, we call it ARIMA controller in the rest of this paper.*

*4. Simulation Study*

*In this section, we will study the performance of the ARIMA controller under multiple scenarios through Monte Carlo simulations. For simplicity, we only focus on the system (1)–(5) with being an ARIMA() disturbance, although the ARIMA controller can be applied to any general ARIMA() disturbance. Without loss of generality, we set the parameters in the ARIMA() disturbance as , , and . For comparison, a controller’s performance is also evaluated. The controller is widely used in feedback control, and it involves two separate constant parameters: the proportional and the integral values, denoted by and . The steps to implement a controller can be found in [25].*

*Tuning a controller could be a challenging task. There are many ways of tuning, such as Ziegler-Nichols tuning, lambda tuning, robust loop shaping, optimization methods, and others (see [26]). Among them, optimization methods are powerful and direct ways. In order to do fair comparisons with the ARIMA controller, we need to choose and to make the controller have the best performance for controlling the specified ARIMA() disturbance. To the best of our knowledge, there are no closed-form expressions for and to optimally control a ARIMA() process; then experimental tuning of the parameters has to be performed. The procedure of optimally choosing and in this paper is described in Appendix B. For the ARIMA() disturbance with and , we get and according to the optimization procedure.*

*Four scenarios are examined in the following simulations. In Scenario 1, the effect of the measurement error and the adjustment error on the ARIMA controller and the controller is investigated; in Scenario 2, the effect of the process initial bias on the two controllers is explored; in Scenario 3, we study how the estimate of the process gain affects the two controllers; in Scenario 4, we investigate both controllers’ performance when the disturbance parameters are not estimated accurately.*

*We do 1000 replications for each case and run 100 steps for in each replication. The first 100-run mean square error (MSE) of is computed. The Average MSE (AMSE) of the 1000 replications is reported in Tables 1–4. Also the standard error in the AMSE (SEAMSE) is computed, which is
where SDMSE is the standard deviation of mean square errors, and the number of replicates here is 1000. AMSE measures the performance of the controllers, and SEAMSE reflects the variability of the AMSE. The smaller the AMSE is, the better performance the controller has.*