Abstract
The dual response surface for simultaneously optimizing the mean and variance models as separate functions suffers some deficiencies in handling the tradeoffs between bias and variance components of mean squared error (MSE). In this paper, the accuracy of the predicted response is given a serious attention in the determination of the optimum setting conditions. We consider four different objective functions for the dual response surface optimization approach. The essence of the proposed method is to reduce the influence of variance of the predicted response by minimizing the variability relative to the quality characteristics of interest and at the same time achieving the specific target output. The basic idea is to convert the constraint optimization function into an unconstraint problem by adding the constraint to the original objective function. Numerical examples and simulations study are carried out to compare performance of the proposed method with some existing procedures. Numerical results show that the performance of the proposed method is encouraging and has exhibited clear improvement over the existing approaches.
1. Introduction
Response surface methodology (RSM) is a design of experimental technique which shows relationship between several designs and response variables. The goal of the experimenter is to determine the optimal settings for the design variables that minimize or maximize the fitted response. For more explanation on response surface techniques see [1–3]. Most of the early work in RSM is centered on a single response problem. This methodology works effectively under the assumption of the homogeneous variance of the response. However, such an assumption may not hold in solving real-life applications. Myers and Carter [4] suggested the need for developing statistical methodology known as dual response surface methodology, which can simultaneously optimize the mean and the variance function as to achieve the desired target while keeping the variance small. Generally, they defined the two responses as primary and secondary. The objective is to find the condition on the design factors that minimize or maximize the primary response function subject to the secondary response . In order to achieve this, three basic strategies are involved: experimental design, regression fitting, and optimization aspect. For the regression fitting, the method of the least squares is usually used to obtain the adequate response functions for the processes mean and variance by assuming that the collected data comes from a normal distribution. While, in the optimization stage, the interest is on what to optimize (i.e., determination of the objective function) and how to optimize (the optimization algorithm), in this paper, we propose a new optimization technique in dual response surface methodology based on the penalty function method for simultaneously optimizing both the location and scale functions. The usefulness of our newly proposed method for estimating the mean and variance of the optimal mean response is studied by some well known data sets and a simulation study. The outline of this paper is organized as follows. In the next section, we discussed some basic concept of the dual response surface followed by the description of the proposed method in Section 3. Numerical examples and simulation study are given in Sections 4 and 5, respectively. Finally, the conclusion is given in Section 6.
2. Dual Response Surface Review
Dual response surface technique consists of finding the optimum setting condition of the controllable factors in order to diminish the performance variability and deviation from the desired target of the decision maker. This method is an extension of the standard ridge analysis procedure which was introduced by Myers and Carter [4]. Ridge analysis has been used by researchers in searching the optimum setting condition for a single response problem [1, 2, 5]. The dual response used mean and variance as separate functions for the system under examination. Then, these functions are optimized based on the chosen optimization technique to determine the optimum operating conditions of the system. Following the strategy of Vining and Myers [6], the mean and the standard deviation fitted response surfaces can be written aswhere wherewhere , , , , , and are estimated vectors and matrices of coefficients obtained from the least squares method. The simultaneous optimization of (1) and (3) using the Lagrangian multipliers method was proposed by Vining and Myers [6]. Lin and Tu [7] noted that the Vining and Myers approach does not always guarantee global optimum solutions due to the restriction of the optimization to equality constraints. Based on this, they proposed the minimization of mean squared error model (MSE) by introducing a slight bias in order to minimize the variability in the responses. This method includes two major parameters (the bias and the variance). Copeland and Nelson [8] observed that minimizing MSE function does not specify how large the estimated mean might be from the specified target value . Instead, they modified VM model by placing some restriction on that minimizes subject to . Kim and Lin [9] introduced a fuzzy modeling methodology using the idea of desirability function method. Several other techniques for solving the dual response surface problem have been presented, for example [10–12] proposed modification of mean squared error model. Furthermore, [13–16] presented a robust design for contaminated and nonnormal data using squared loss optimization scheme, a highly efficient and outlier resistant robust design estimator, a dual response approach to multiple response robust design problem, a multivariate robust design using MSE and dual response modeling, and robust parameter design, respectively. Recently, a biobjective robust design model has been developed in [17] and a robust cutting parameter design using computer simulation has been studied in [18]. In many real world situations, the experimenter or the decision maker often needs to keep a balance between the process mean and the process variance to achieve the desired target. It is known that getting all efficient solutions with the class of LT methods is challenging due to the large resulting process variance. However, most of the preceding optimization schemes are derived from the LT model, except the Vining and Myers model which is basically based on the Lagrangian multipliers approach. In this paper, an alternative objective function is considered based on the penalty function method.
3. Proposed Optimization Scheme for Dual Response Surface
In the present study, we present a new optimization technique for dual response surface methodology based on the penalty function method. The penalty function approach swaps a constrained optimization problem by a sequence of unconstrained optimization problems whose approximate solution ideally converges to a true solution of the original constrained problem. The unconstrained problem is formulated by adding a penalty term to the original objective function which consists of the penalty parameter multiplied by a measure of violation of the constraints [19, 20]. Consider the general formulation of the constrained optimization problem given below:By applying the penalty function method, we can obtain the solution of (5) using the modified objective function:where is the original objective function to be minimized and and are set of inequality and equality constraints, respectively. This paper specifically considers a quadratic penalty function of the formwhere is called the penalty constant which penalizes the equality constraints when the constraints relations are not satisfied. For the purpose of clarity, we replaced and in (7) with and and then write the following quadratic unconstrained minimization problem aswhere is the fitted response surface for mean, is the fitted response surface for the standard deviation function, and is the target value (usually specified by experimenter). If , the method gives exact solution. Since in this case it is necessary that which implies the bigger penalty parameter , thus the more exact solution is achieved. For (8), we can apply any unconstrained optimization method such as Newton’s method, BFGS method, Conjugate gradient method, and Steepest ascent (descent) method. Moreover, any nonlinear optimization software may be used to find the optimal design settings for the dual response surface problem. We used the package Rsolnp introduced in [21, 22] in R language, which is open source statistical software to perform the numerical computations and analysis. Our aim is to find an optimum solution such that the estimated mean value will be very close or equal to the target value , while the variance is kept small. The proposed approach has some advantages over some existing methods. Firstly, the proposed method takes into consideration the measure of violation of the constraint, whereas the VM [6] and the class of LT methods are minimized without regard to the relative magnitude of violation of constraint. Secondly, the penalty parameter in (8) forced the to be close to zero or equal to zero so as to achieve the target of the experimenter (decision maker). Therefore, we anticipate the optimal setting condition obtained by the proposed method would be more efficient in terms of the contribution of both bias and variance components of of the estimated mean response compared with other existing methods.
4. Simulation Study and Results
In this section, a simulation study is conducted to assess the performance of the newly proposed method and compare it with the commonly used methods, such as VM, LT, and WMSE. Following [13, 23] the five responses are randomly generated from a normal distribution with mean and standard deviation at each control factor of settings , . The mean and the standard deviation are given aswhere . All the four methods were then applied to the data. The total of 500, 1000, and 2000 iterations is considered. Some summary values, such as the estimated mean of the optimal mean response, computed over iterations are defined by , bias = , and . The mean squared error, denoted by (MSE), is written as . Hence, the root mean squared error (RMSE) is given by . The bias, standard error, and root mean squared error (RMSE) of the estimates of the optimal mean response are exhibited in Table 1.
Figures 1 and 2 show the estimated bias and mean squared error based on the total number iteration for the various methods. It can be observed that the bias of the VM estimate is smaller than the LT and WMSE estimates. However, its RMSE is the largest among the three estimates since the variance of the VM estimate makes up most of the MSE. It is interesting to see that our proposed method is the best in terms of the smallest bias and RMSE values. Due to space constraint, Figure 3 presents kernel density estimates of VM, LT, WMSE, and PM for 1000 iterations only. The plotted results indicate that the proposed approach has a good behavior in which it is very close to the desired target. Therefore, one can say that the behavior of the constructed objective function based on the penalty function technique is more efficient and robust than other existing methods for solving dual response surface optimization problem.
5. Numerical Examples
5.1. Printing Process Study Data
To show a clear comparison, we consider the data set used by Vining and Myers [6] and Lin and Tu [7] which is given in Table 2. The experiment was conducted to determine the effect of the three variables (speed), (pressure), and (distance) on the quality of the printing process, that is, on the machines ability to apply colored inks to package labels. The experiment is a factorial design with 3 replicates at each point. Firstly, the average and variance of the 3 responses at each design point are computed, respectively. Vining and Myers [6] used the least squares method to fit a quadratic response surface model for mean and standard deviation as follows:Based on the models in (10), Table 3 gives the summary of the results for the four different approaches obtained using the cuboidal region , . The optimal setting, estimated mean response, estimated standard deviation, and RMSE are presented in Table 3. The RMSE is calculated using the formula , where . It can be seen that the VM approach leads to the optimum setting which resulted in an estimated mean response of (501.57) and root mean squared error of (51.94). This optimal setting is very close to the target mean response but with larger RMSE. The second approach LT produces a target value of (494.69) and RMSE of (44.78) which is smaller than the RMSE of the VM. The third approach is the WMSE which gives the estimated mean response of (496.44) and RMSE of (44.81) which indicate a slight increase in RMSE and little improvement in mean response. The overall performance of the proposed approach is better than those three approaches mentioned with approximate mean value of (500.00) and RMSE of (44.75). Similar procedure is repeated in the next example in order to demonstrate a clear advantage of using the proposed method in terms of closeness to target mean response and the smallest RMSE.
5.2. The Catapult Study Data
This example will consider the data used by Luner [24] and Kim and Lin [9]. Three variables, (arm length), (stop angle), and (pivot height), are under consideration to predict the distance to the point where a projectile landed from the base of the roman style catapult. The experiment is a central composite design with three replicates as given in Table 5. In [9], the fitted second order polynomial regression models for the mean and standard deviation functions are given byHere, the assumed target mean value is . Table 4 shows the estimated mean and RMSE of the estimated optimal mean response. It is evident from Table 4 that the proposed method outperformed the other existing procedures.
6. Conclusion
Numerous procedures have been developed in the literature to obtain an optimal setting condition for the dual response methodology. This paper discusses four different objective functions for dual response optimization approach, based on the mean and variance models as separate response functions. The proposed objective function is based on the penalty function method. We have proposed a new objective function which is more efficient compared with the other existing methods. Numerical examples and simulations study are carried out to compare the performance of the newly proposed method with the frequently used methods. The numerical results clearly show an improvement of the proposed method over the existing methods in terms of having the smallest bias and RMSE. Moreover, the proposed approach can be applied to ridge analysis method and robust parameter design optimization.
Abbreviations
| : | Desired target for the mean response |
| : | Vector of control variables |
| : | Vector of the observed responses |
| : | Standard deviation of the observed responses |
| : | Penalty constant |
| : | Desired upper bound for the bias |
| : | Estimated mean of the optimal response |
| : | Average of the observed responses |
| : | Fitted response surface for the mean function |
| : | Fitted response surface for the standard deviation function |
| VM: | Vining and Myers [6] |
| PM: | Proposed method in this paper |
| LT: | Lin and Tu or can be referred to as MSE [7] |
| WMSE: | Ding et al. [10] |
| BFGS: | Broyden-Fletcher-Goldfarb-Shanno |
| MSE: | Mean squared error of the estimated mean optimal response |
| : | Root Mean squared error of the estimated mean optimal response. |
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.