- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
Advances in Artificial Neural Systems
Volume 2013 (2013), Article ID 628313, 9 pages
Desirability Improvement of Committee Machine to Solve Multiple Response Optimization Problems
1Department of Mechanical and Manufacturing Engineering, Universiti Putra Malaysia, 43400 Serdang, Selangor, Malaysia
2Faculty of Engineering, Tarbiat Modares University, P.O. Box 14155-143, Tehran 1411713116, Iran
Received 28 February 2013; Accepted 8 August 2013
Academic Editor: Kyong Joo Oh
Copyright © 2013 Seyed Jafar Golestaneh et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Multiple response optimization (MRO) problems are usually solved in three phases that include experiment design, modeling, and optimization. Committee machine (CM) as a set of some experts such as some artificial neural networks (ANNs) is used for modeling phase. Also, the optimization phase is done with different optimization techniques such as genetic algorithm (GA). The current paper is a development of recent authors' work on application of CM in MRO problem solving. In the modeling phase, the CM weights are determined with GA in which its fitness function is minimizing the RMSE. Then, in the optimization phase, the GA specifies the final response with the object to maximize the global desirability. Due to the fact that GA has a stochastic nature, it usually finds the response points near to optimum. Therefore, the performance the algorithm for several times will yield different responses with different GD values. This study includes a committee machine with four different ANNs. The algorithm was implemented on five case studies and the results represent for selected cases, when number of performances is equal to five, increasing in maximum GD with respect to average value of GD will be eleven percent. Increasing repeat number from five to forty-five will raise the maximum GD by only about three percent more. Consequently, the economic run number of the algorithm is five.
Multiple response optimization (MRO) problems need to find a set of input variable values (x’s) which get a desired set of outputs (y's). The current study develops a proposed algorithm in recent authors' work to solve MRO problems . MRO solution methodologies usually include three phases: experiments design, modeling, and optimization.
There are some techniques for experiments design. Some methodologies in this phase are as follows: design of experiments (DOEs) knowledge such as factorial design and fraction factorial design, response surface methodology (RSM) such as central composite design (CCD), and Box Behnken [2, 3]. Furthermore, Taguchi orthogonal arrays [4–7] are derived from the Taguchi method.
Modeling as the second phase is done using different mathematical or statistical models such as multiple linear and nonlinear regressions in the form of polynomials [2, 8, 9] and artificial neural networks (ANNs). Due to the existence of complicated relationship between inputs and outputs, usually ANNs are mostly used for modeling rather than for polynomials. One famous artificial neural network (ANN) is back propagation neural network (BPNN) that is used in many engineering problems [10, 11]. Cheng et al.  utilized MANFIS (multiadaptive neuro fuzzy inference system) for modeling and showed that the results are superior to RSM polynomial models.
The last phase is optimization, which is usually done on a performance metric such as global desirability function. In this process, each predicted response is converted to a value between 0 and 1. Finally, a composite function is defined which converts all desirability functions to a unique number by global desirability function (GDF). Also, Chatsirirungruang and Miyakawa  proposed a combination of Taguchi and GA to get more accurate responses by using the benefits of both techniques together.
2. Neural Networks and Committee Machine
Different kinds of neural networks are used to model in complicated prediction problems. Four neural networks are used in this study that include feed forward neural networks (FF) , radial basis function networks (RBFNs) , generalized regression neural network (GRNN) , and adaptive neural fuzzy inference system (ANFIS) [17, 18].
A committee machine (CM) is a collection of some intelligent systems named experts and a combiner which combines the outputs of each expert (Figure 1). The advantage of CM is that it reaps the benefits of all work with only little additional computation. Independent variables are entered for experts, and all experts' outputs are transferred to a combiner to get the final response.
One of the most popular methods to combine the experts’ outputs is the simple ensemble averaging method according to (1) . Furthermore, a combiner could be an intelligent system such as a neural network. Consider where is the weight coefficient of th expert, is the estimated response from th expert, and is the total number of the experts .
Genetic algorithm could be used to yield the experts' contribution (weights) in a committee machine. Equation (2) represents that the committee machine gives smaller errors than the average of all the experts [20, 21]: where is the error of predicted and real response of each expert and is the squared error for the th expert. Errorave is the average error for all experts and ErrorCM is the error of CM.
3. Global Desirability and Genetic Algorithm
Overall, desirability or global desirability function is used to transmit multiple responses to a single response case. Desirability function converts each estimated response into a dimensionless desirability value . It gets values according to the kind of objects in the problem. These conditions are shown in (3), (4), and (5) [31, 32].
3.1. Desirability Functions Formula with Different Objects
The desirability for goal of “Target:” ?The desirability for goal of “Maximum:” ?The desirability for goal of “Minimum:” where the parameters and in the formulas are convexity coefficients and specify how strictly the target value will be desired. In the current study, and are equal to one. Global desirability (GD) function is according to (6): Equations from (3) to (5), yield the single desirabilities for different objects and (6) calculates the global desirability (GD). Both ’s and GD values range vary from zero to one. In the MRO problems, it is important that all responses optimize simultaneously, and GD is a suitable performance metric to achieve this target.
Genetic algorithm (GA) is a population-based search technique, which can quickly and reliably solve problems that are difficult to tackle by traditional methods. One advantage of GA is that it is extensible and can interface with existing models and hybridize with them and optimizes the fitness function [33, 34].
Also Brie and Morignot  state that genetic algorithm has stochastic nature, and consequently, the results may highly vary from test to test, even for the same problem and parameter set.
Different methods have been proposed in the literature for the optimization of multiple response problems. Table 1 shows corresponding techniques. In this table, some include only investigation for analysis and comparison not optimization.
As a consequence, by reviewing the above works and other works in the literature, since the genetic algorithm has been widely implemented by the researchers for optimization phase of MRO problems with respect to other techniques, this metaheuristic algorithm was selected as the optimization technique.
First of all, an important matter is the selection of data for training and testing of model. Dixit and Chandra  have suggested a selection method for ANNs. According to their suggestions, for n inputs, the minimum number of training set should be such that it includes the corners of n-dimensional space with respect to more contribution to input variables with more influence on output. In the current investigation, this suggestion was applied for corners of lower and upper limits for all independent variables. Also, training and testing dataset numbers were 80 and 20 percent, respectively.
Different criteria are used to assess forecasting models performance. Two criteria were selected in the current work, which compare models’ results with the observed or real data. They are root mean square error (RMSE)  and correlation coefficient () . Consider where is th predicted value or model output, is the th actual value, and is the number of data used for prediction. Furthermore, and are the means of actual and predicted values . The current work includes two conditions to build ANNs model: first is that RMSE for all data is the minimum and the second condition is that the correlation coefficient of testing data is positive.
Usually, MRO solution includes three phases. Phase one is experiments design, in which in the current work, all data are selected from the literatures. The second phase is modeling which is done by building four different neural networks and a committee machine. ANNs include feed forward, RBF, GRNN, and ANFIS models. All neural networks have the same inputs and one output, and so the number of ANNs in each model is equal to the number of responses (Figure 2) .
A committee machine (CM) was made by a combination of all four ANN models (Figure 3). inputs are entered for each expert of CM simultaneously, and responses are multiplied to their weights and then are added together to get the final response. Committee machine combiner is an ensemble averaging. Genetic algorithm (GA) computes CM weights with the object to minimize RMSE of CM response. So the weight matrix is an matrix.
The object of the current study is to find the economic performance number of the committee machine and genetic algorithm to get the best responses in MRO problems solving. Therefore, firstly, four ANNs and one committee machine were created separately. Committee machine weights were calculated by means of GA with the object of minimizing overall RMSE. Then in the optimization phase, GA yields the best responses with the object of maximizing global desirability. The result is and with the highest possible GD. These calculations of finding CM weights and were repeated 45 times.
5. Results and Discussion
Genetic algorithm is applied in two steps. The first step is to find CM weights with the object of minimizing the overall RMSE of CM, and the second step is to find the ’s by GA and ANNs with the object of maximizing global desirability. In both steps, GA specifications are listed in Table 2.
The current algorithm is implemented on five MRO problems. These problems include different numbers of inputs and outputs and different numbers of experiments. Table 3 represents their properties.
Case 1. The first problem is based on the wire-bonding process in the semiconductor industry. Table 4 represents the process inputs and outputs. Different neural networks were established to model data of experiments. Table 5 lists the ANNs specifications for Case 1. For other cases, to have superior comparison between committee machine and other neural networks, the same specifications were considered according to Table 6. Case 3 has deferent specifications to get acceptable results. Four neural networks that include feedforward (FF), radial base function (RBF), GRNN, and ANFIS were consisted in each response for each problem data. So every problem finds ( no. of responses) models. A committee machine was set with the object to minimize the overall RMSE.
Case 2. The problem is to optimize the yield of recombinant Oryza sativa nonsymbiotic hemoglobin 1 in a medium containing byproduct glycerol. Table 7 represents the input and output variables of this case.
Case 3. The problem is multiple response optimization of styrene-butadiene rubber (SBR) emulsion batch polymerization. Table 8 lists the input and output variables.
Case 4. The object of this case is to optimize process variables, electrolysis voltage, and treatment time for the electrocoagulation removal of hexavalent chromium (Cr(VI)). Table 9 represents the input and output variables.
Case 5. The problem is to optimize multiple characteristics in CNC turning of AISI P-20 tool steel using liquid nitrogen as a coolant. Table 10 lists the input and output variables.
In all five cases, the CM responses that include GD and RMSE were calculated 45 times. The results of GD are listed in Tables 11 and 12, representing the statistical results. Case 1 was eliminated in the calculations and the reason is due to the existence of zero values in GD; the increasing of maximum GD to average GD is very high and this can mislead us to unmoral results. So only cases from two to five are considered and this will yield smaller increase, but more reliable.
The GD ratio is defined in formula (8) and represents ratio of increasing maximum GD to average GD: Also, to investigate for ANNs behavior, the results of five runs are listed in Table 13. For abstract only Case 2 is listed. Table 14 represents statistical results of this case.
It is obvious that in both CM and ANNs, RMSE is constant for all run numbers. Table 13 shows this reality for Case 2 with ANNs models. Table 13 shows there is no significant difference between GD values with respect to run numbers for different ANNs runs. Table 13 represents, for all case, that there is an increase in the mean of GD ratio (or mean of increasing the maximum GD to average GD) with respect to increasing the run number.
Figure 5 shows the corresponding results graphically and it illustrates that for committee machine, if the program performs, for example, 3 times, the maximum to average will increase to 7.8%. In addition, it shows that if the program runs 5 times, the maximum value of GD can increase to 11.7% with respect to average. From run numbers 5 to 8, there is a slight rise about 1.6%. From run numbers 8 to 10, there is a relatively fall in GD ratio. Then from run numbers ten to forty-five, there is no significant rise in GD ration and it is only 2.3 percent (from 12.8% to 15,1%). Consequently, the economical run number for the algorithm is five times. Because by consuming time from 5 to 45 times will increase GD ratio about 3.2% (15.1%–11.7%) whereas run number equal five times has 11.7% and more than 3 times.
Table 13 shows for different ANNs run numbers, there is only about 1% increasing in GD ratio for run numbers more than one and this rise is not noticeable, because increasing 1% is due to nature of GA. So to run more than 3 times for neural networks models has no noticeable effect to increase GD ratio.
Multiple response optimization (MRO) problem solving is usually done in three phases that include experiments design, modeling, and optimization. Committee machine (CM) as a collection of some experts such as some artificial neural networks (ANNs) can be used in the modeling phase of MRO. Genetic algorithm is used to find CM weights in the modeling phase and also as main optimization techniques in the optimization phase.
The current study modifies a proposed algorithm from recent works of authors that had used CM and GA to solve MRO problems. Due to stochastic nature of GA, the final solutions vary together and different performances will yield different responses with related global desirability (GD). So since object of MRO is to find responses with highest GD, to know economic run number will be useful to obtain best responses in minimum possible time. According to this investigation and for the selected MRO problems, the results represent that the economic run number of the algorithm is five. With five run numbers, maximum global desirability of final solution can increase about 11 percent in concern with average of GD. Whereas, to run the algorithm from five to forty-five numbers, the maximum of global desirability can increase only about 3 percent more.
The authors wish to thank Sadid Industrial Group and especially Dr. A. A. Maghsoudi for the support.
- S. J. Golestaneh, N. Ismail, S. H. Tang, M. K. A. M. Ariffin, H. Moslemi Naeini, and A. A. Maghsoudi, “A committee machine approach to multiple response optimization,” International Journal of Physical Sciences, vol. 6, no. 35, pp. 7935–7949, 2011.
- E. Del Castillo, D. C. Montgomery, and D. R. McCarville, “Modified desirability functions for multiple response optimization,” Journal of Quality Technology, vol. 28, no. 3, pp. 337–345, 1996.
- W. Guo, Y. Zhang, J. Lu et al., “Optimization of fermentation medium for nisin production from Lactococcus lactis subsp. lactis using response surface methodology (RSM) combined with artificial neural network-genetic algorithm (ANN-GA),” African Journal of Biotechnology, vol. 9, no. 38, pp. 6264–6272, 2010.
- J. Antony, R. B. Anand, M. Kumar, and M. K. Tiwari, “Multiple response optimization using Taguchi methodology and neuro-fuzzy based model,” Journal of Manufacturing Technology Management, vol. 17, no. 7, pp. 908–925, 2006.
- H. Chang, “A data mining approach to dynamic multiple responses in Taguchi experimental design,” Expert Systems with Applications, vol. 35, no. 3, pp. 1095–1103, 2008.
- S. Kumanan, J. E. R. Dhas, and K. Gowthaman, “Determination of submerged arc welding process parameters using Taguchi method and regression analysis,” Indian Journal of Engineering and Materials Sciences, vol. 14, no. 3, pp. 177–183, 2007.
- A. W. L. Yao, H. T. Liao, and C. Y. Liu, “A taguchi and neural network based electric load demand forecaster,” The Open Automation and Control System Journal, vol. 1, pp. 7–13, 2008.
- D. Lepadatu, A. Kobi, R. Hambli, and A. Barreau, “Lifetime multiple response optimization of metal extrusion die,” in Proceedings of the Annual Reliability and Maintainability Symposium (RAMS '05), pp. 37–42, January 2005.
- S. H. R. Pasandideh and S. T. A. Niaki, “Multi-response simulation optimization using genetic algorithm within desirability function framework,” Applied Mathematics and Computation, vol. 175, no. 1, pp. 366–382, 2006.
- I. Mukherjee and P. K. Ray, “A modified tabu search strategy for multiple-response grinding process optimisation,” International Journal of Intelligent Systems Technologies and Applications, vol. 4, no. 1-2, pp. 97–122, 2008.
- R. Noorossana, S. Davanloo Tajbakhsh, and A. Saghaei, “An artificial neural network approach to multiple-response optimization,” International Journal of Advanced Manufacturing Technology, vol. 40, no. 11-12, pp. 1227–1238, 2009.
- C. Cheng, C.-J. Cheng, and E. S. Lee, “Neuro-fuzzy and genetic algorithm in multiple response optimization,” Computers and Mathematics with Applications, vol. 44, no. 12, pp. 1503–1514, 2002.
- P. Chatsirirungruang and M. Miyakawa, “Application of genetic algorithm to numerical experiment in robust parameter design for signal multi-response problem,” International Journal of Management Science and Engineering Management, vol. 4, no. 1, pp. 49–59, 2009.
- T. Kamo and C. Dagli, “Hybrid approach to the Japanese candlestick method for financial forecasting,” Expert Systems with Applications, vol. 36, no. 3, pp. 5023–5030, 2009.
- H. B. Celikoglu, “Application of radial basis function and generalized regression neural networks in non-linear utility function specification for travel mode choice modelling,” Mathematical and Computer Modelling, vol. 44, no. 7-8, pp. 640–658, 2006.
- Matlab User’s Guide: Neural Network Toolbox Version 2010, MathWorks, Natick, MA, USA, 2010.
- E. Ardil and P. S. Sandhu, “A soft computing approach for modeling of severity of faults in software systems,” International Journal of Physical Sciences, vol. 5, no. 2, pp. 074–085, 2010.
- J. Bo, T. Yuchun, and Z. Yan-Qing, “Hybrid SVM-ANFIS for protein subcellular location prediction,” International Journal of Computational Intelligence in Bioinformatics and Systems Biology, vol. 1, no. 1, p. 59, 2009.
- N. Ismail, S. J. Golestaneh, S. H. Tang et al., “Modified committee neural networks for prediction of machine failure times,” in Proceedings of the 3rd national intelligent systems and information technology symposium (ISITS '10), 2010.
- A. Kadkhodaie-Ilkhchi, M. R. Rezaee, and H. Rahimpour-Bonab, “A committee neural network for prediction of normalized oil content from well log data: an example from South Pars Gas Field, Persian Gulf,” Journal of Petroleum Science and Engineering, vol. 65, no. 1-2, pp. 23–32, 2009.
- S. Karimpouli, N. Fathianpour, and J. Roohi, “A new approach to improve neural networks' algorithm in permeability prediction of petroleum reservoirs using supervised committee machine neural network (SCMNN),” Journal of Petroleum Science and Engineering, vol. 73, no. 3-4, pp. 227–232, 2010.
- K. Y. Benyounis, A. G. Olabi, and M. S. J. Hashmi, “Multi-response optimization of CO2 laser-welding process of austenitic stainless steel,” Optics and Laser Technology, vol. 40, no. 1, pp. 76–87, 2008.
- C. Cojocaru, M. Khayet, G. Zakrzewska-Trznadel, and A. Jaworska, “Modeling and multi-response optimization of pervaporation of organic aqueous solutions using desirability function approach,” Journal of Hazardous Materials, vol. 167, no. 1–3, pp. 52–63, 2009.
- G. Martinez Delfa, A. Olivieri, and C. E. Boschetti, “Multiple response optimization of styrene-butadiene rubber emulsion polymerization,” Computers and Chemical Engineering, vol. 33, no. 4, pp. 850–856, 2009.
- D. S. Nagesh and G. L. Datta, “Genetic algorithm for optimization of welding variables for height to width ratio and application of ANN for prediction of bead geometry for TIG welding process,” Applied Soft Computing Journal, vol. 10, no. 3, pp. 897–907, 2010.
- A. Patnaik and S. Biswas, “An evolutionary approach to parameter optimisation of submerged arc welding in the hardfacing process,” International Journal of Manufacturing Research, vol. 2, no. 4, pp. 462–483, 2007.
- C. Pizarro, J. M. González-Sáiz, and N. Pérez-del-Notario, “Multiple response optimisation based on desirability functions of a microwave-assisted extraction method for the simultaneous determination of chloroanisoles and chlorophenols in oak barrel sawdust,” Journal of Chromatography A, vol. 1132, no. 1-2, pp. 8–14, 2006.
- P. C. Giordano, H. D. Martínez, A. A. Iglesias, A. J. Beccaria, and H. C. Goicoechea, “Application of response surface methodology and artificial neural networks for optimization of recombinant Oryza sativa non-symbiotic hemoglobin 1 production by escherichia coli in medium containing byproduct glycerol,” Bioresource Technology, vol. 101, no. 19, pp. 7537–7544, 2010.
- M. S. Bhatti, A. S. Reddy, R. K. Kalia, and A. K. Thukral, “Modeling and optimization of voltage and treatment time for electrocoagulation removal of hexavalent chromium,” Desalination, vol. 269, no. 1–3, pp. 157–162, 2011.
- A. Aggarwal, H. Singh, P. Kumar, and M. Singh, “Optimization of multiple quality characteristics for CNC turning under cryogenic cutting environment using desirability function,” Journal of Materials Processing Technology, vol. 205, no. 1-3, pp. 42–50, 2008.
- K. Y. Benyounis and A. G. Olabi, “Optimization of different welding processes using statistical and numerical approaches—a reference guide,” Advances in Engineering Software, vol. 39, no. 6, pp. 483–496, 2008.
- H. Chang and Y. Chen, “Neuro-genetic approach to optimize parameter design of dynamic multiresponse experiments,” Applied Soft Computing Journal, vol. 11, no. 1, pp. 436–442, 2011.
- Y. Liang, “Combining neural networks and genetic algorithms for predicting the reliability of repairable systems,” International Journal of Quality and Reliability Management, vol. 25, no. 2, pp. 201–210, 2008.
- L. Tian and A. Noore, “Evolutionary neural network modeling for software cumulative failure time prediction,” Reliability Engineering and System Safety, vol. 87, no. 1, pp. 45–51, 2005.
- A. H. Brie and P. Morignot, “Genetic planning using variable length chromosomes,” in Proceedings of the International Conference on Automated Planning and Scheduling/Artificial Intelligence Planning Systems (ICAPS/AIPS '05)., 2005.
- U. S. Dixit and S. Chandra, “A neural network based methodology for the prediction of roll force and roll torque in fuzzy form for cold flat rolling process,” International Journal of Advanced Manufacturing Technology, vol. 22, no. 11-12, pp. 883–889, 2003.
- A. Haghizadeh, L. T. Shui, and E. Goudarzi, “Estimation of yield sediment using artificial neural network at basin scale,” Australian Journal of Basic and Applied Sciences, vol. 4, no. 7, pp. 1668–1675, 2010.
- P. Krause, D. P. Boyle, and F. Bäse, “Comparison of different efficiency criteria for hydrological model assessment,” Advances in Geosciences, vol. 5, pp. 89–97, 2005.
- S. Banik, M. Anwer, and A. F. M. K. Khan, “Predictive power of the daily Bangladeshi exchange rate series based on markov model, neuro fuzzy model and conditional heteroskedastic model,” in Proceedings of the 12th International Conference on Computer and Information Technology (ICCIT '09), pp. 303–308, December 2009.