• Views 1,001
• Citations 1
• ePub 36
• PDF 455
`Advances in Artificial Neural SystemsVolume 2013 (2013), Article ID 284570, 17 pageshttp://dx.doi.org/10.1155/2013/284570`
Research Article

## Variance Sensitivity Analysis of Parameters for Pruning of a Multilayer Perceptron: Application to a Sawmill Supply Chain Simulation Model

Centre de Recherche en Automatique de Nancy (CRAN-UMR 7039), Nancy-Université, CNRS, Campus Sciences, BP 70239, 54506 Vandoeuvre les Nancy Cedex, France

Received 6 May 2013; Revised 15 July 2013; Accepted 15 July 2013

Copyright © 2013 Philippe Thomas et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. P. Lopez and F. Roubellat, Ordonnancement de la Production, Hermès, Paris, France.
2. E. Goldratt and J. Cox, The Goal: A Process of Ongoing Improvement, The North River Press, Great Barrington, Mass, USA, 2nd edition, 1992.
3. A. Thomas and P. Charpentier, “Reducing simulation models for scheduling manufacturing facilities,” European Journal of Operational Research, vol. 161, no. 1, pp. 111–125, 2005.
4. E. H. Page, D. M. Nicol, O. Balci et al., “An aggregate production planning framework for the evaluation of volume flexibility,” in Proceedings of the Winter Simulation Conference (WSC '99), pp. 1509–1520, Phoenix, Ariz, USA, December 1999.
5. S. C. Ward, “Argument for constructively simple models,” Journal of the Operational Research Society, vol. 40, no. 2, pp. 141–153, 1989.
6. R. J. Brooks and A. M. Tobias, “Simplification in the simulation of manufacturing systems,” International Journal of Production Research, vol. 38, no. 5, pp. 1009–1027, 2000.
7. L. Chwif, R. J. Paul, and M. R. P. Barretto, “Discrete event simulation model reduction: a causal approach,” Simulation Modelling Practice and Theory, vol. 14, no. 7, pp. 930–944, 2006.
8. P. Thomas and A. Thomas, “Expérimentation de la reduction d'un modèle de simulation par réseau de neurones: cas d'une scierie,” in Proceedings of the 7ème Conférence Internationale de Modélisation, Optimisation et Simulation des Systèmes (MOSIM '08), pp. 2030–2038, Paris, France, March-April 2008.
9. P. Thomas, G. Bloch, F. Sirou, and V. Eustache, “Neural modeling of an induction furnac using robust learning criteria,” Integrated Computer-Aided Engineering, vol. 6, no. 1, pp. 15–26, 1999.
10. P. Thomas, D. Choffel, and A. Thomas, “Simulation reduction models approach using neural network,” in Proceedings of the 10th International Conference on Computer Modelling and Simulation (EUROSIM '08), Cambridge, UK, April 2008.
11. P. L. Bartlett, “For valid generalization, the size of the weights is more important than the size of the network,” Neural Information Processing Systems, vol. 9, pp. 134–140, 1997.
12. G. C. Cawley and N. L. C. Talbot, “Preventing over-fitting during model selection via bayesian regularisation of the hyper-parameters,” Journal of Machine Learning Research, vol. 8, pp. 841–861, 2007.
13. H. Drucker, “Effect of pruning and early stopping on performance of a boosting ensemble,” Computational Statistics and Data Analysis, vol. 38, no. 4, pp. 393–406, 2002.
14. C. M. Bishop, Neural Network for Pattern Recognition, Oxford University Press, Oxford, UK, 1995.
15. P. Lauret, E. Fock, and T. A. Mara, “A node pruning agorithm based on a fourier amplitude sensitivity test method,” IEEE Transactions on Neural Networks, vol. 17, no. 2, pp. 273–293, 2006.
16. R. Chentouf and C. Jutten, “Combining sigmoids and radial basis function in evolutive neural architecture,” in Proceedings of the 4th European Symposium on Artificial Neural Networks (ESANN '96), pp. 129–134, Bruges, Belgium, April 1996.
17. R. Setiono, “Feedforward neural network construction using cross validation,” Neural Computation, vol. 13, no. 12, pp. 2865–2877, 2001.
18. I. Rivals and L. Personnaz, “Neural-network construction and selection in nonlinear modeling,” IEEE Transactions on Neural Networks, vol. 14, no. 4, pp. 804–819, 2003.
19. L. Ma and K. Khorasani, “New training strategies for constructive neural networks with application to regression problems,” Neural Networks, vol. 17, no. 4, pp. 589–609, 2004.
20. B. Hassibi and D. G. Stork, “Second order derivatives for network pruning: optimal brain surgeon,” in Advances in Neural Information Processing Systems, S. H. Hanson, J. D. Cowan, and C. L. Gilles, Eds., pp. 164–171, Morgan Kaufmann, San Mateo, Calif, USA, 1993.
21. M. Cottrell, B. Girard, Y. Girard, M. Mangeas, and C. Muller, “Neural modeling for time series: a statistical stepwise method for weight elimination,” IEEE Transactions on Neural Networks, vol. 6, no. 6, pp. 1355–1364, 1995.
22. P. Thomas and G. Bloch, “Robust pruning for multilayer perceptrons,” in Proceedings of the 2nd IMACS/IEEE Multiconference on Computational Engineering in Systems Applications (CESA '98), pp. 17–22, Nabeul-Hammamet, Tunisia, April 1998.
23. J. Xu and D. W. C. Ho, “A new training and pruning algorithm based on node dependence and Jacobian rank deficiency,” Neurocomputing, vol. 70, no. 1–3, pp. 544–558, 2006.
24. X. Zeng and D. S. Yeung, “Hidden neuron pruning of multilayer perceptrons using a quantified sensitivity measure,” Neurocomputing, vol. 69, no. 7–9, pp. 825–837, 2006.
25. X. Liang, “Removal of hidden neurons in multilayer perceptrons by orthogonal projection and weight crosswise propagation,” Neural Computing and Applications, vol. 16, no. 1, pp. 57–68, 2007.
26. E. Romero and J. M. Sopena, “Performing feature selection with multilayer perceptrons,” IEEE Transactions on Neural Networks, vol. 19, no. 3, pp. 431–441, 2008.
27. I. T. Jollife, Principal Component Analysis, Springer, New York, NY, USA, 1986.
28. P. Demartines, Analyse de données par réseaux de neurones auto-organisés [Ph.D. dissertation], Institut National Polytechnique de Grenoble, Valence, France, 1995.
29. G. Dreyfus, J. M. Martinez, M. Samuelides et al., Réseaux de Neurones: Méthodologies et Applications, Editions Eyrolles, Paris, France, 2002.
30. H. Stoppiglia, G. Dreyfus, R. Dubois, and Y. Oussar, “Ranking a random feature for variable and feature selection,” Journal of Machine Learning Research, vol. 3, pp. 1399–1414, 2003.
31. T. Cibas, F. F. Soulié, P. Gallinari, and S. Raudys, “Variable selection with neural networks,” Neurocomputing, vol. 12, no. 2-3, pp. 223–248, 1996.
32. P. Leray and P. Gallinari, “Feature selection with neural networks,” Behaviometrika, vol. 26, no. 1, pp. 145–166, 1999.
33. G. Castellano and A. M. Fanelli, “Variable selection using neural-network models,” Neurocomputing, vol. 31, no. 1–4, pp. 1–13, 2000.
34. S. Gadat and L. Younes, “A stochastic algorithm for feature selection in pattern recognition,” Journal of Machine Learning Research, vol. 8, pp. 509–547, 2007.
35. R. Setiono and W. K. Leow, “Pruned neural networks for regression,” in Proceedings of the 6th Pacific Rim International Conference on Artificial Intelligence (PRICAI '00), pp. 500–509, Melbourne, Australia, August 2000.
36. A. P. Engelbrecht, “A new pruning heuristic based on variance analysis of sensitivity information,” IEEE Transactions on Neural Networks, vol. 12, no. 6, pp. 1386–1389, 2001.
37. B. P. Zeigler, Theory of Modeling and Simulation, John Wiley & Sons, New York, NY, USA, 1976.
38. G. Innis and E. Rexstad, “Simulation model simplification techniques,” Simulation, vol. 41, no. 1, pp. 7–15, 1983.
39. R. C. Leachman, Preliminary Design and Development of a Corporate Level Production Planning System for the Semi Conductor Industry, Optimization in Industry, Chichester, UK, 1986.
40. Y.-F. Hung and R. C. Leachman, “Reduced simulation models of wafer fabrication facilities,” International Journal of Production Research, vol. 37, no. 12, pp. 2685–2701, 1999.
41. S. Hsieh, J.-S. Hwang, and H.-C. Chou, “A petri-net-based structure for AS/RS operation modelling,” International Journal of Production Research, vol. 36, no. 12, pp. 3323–3346, 1998.
42. G. Cybenko, “Approximation by superpositions of a sigmoidal function,” Mathematics of Control, Signals, and Systems, vol. 2, no. 4, pp. 303–314, 1989.
43. K.-I. Funahashi, “On the approximate realization of continuous mappings by neural networks,” Neural Networks, vol. 2, no. 3, pp. 183–192, 1989.
44. R. Reed, “Pruning algorithms—a survey,” IEEE Transactions on Neural Networks, vol. 4, no. 5, pp. 740–747, 1993.
45. C. Jutten and O. Fambon, “Pruning methods: a review,” in Proceedings of the 3rd European Symposium on Artificial Neural Networks (ESANN '95), pp. 129–140, Brussels, Belgium, April 1995.
46. Y. LeCun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” Advances in Neural Information Processing Systems, vol. 2, pp. 598–605, 1990.
47. M. Norgaard, System identification and control with neural networks [Ph.D. dissertation], Institute of Automation, Technical University of Denmark, Copenhagen, Denmark, 1996.
48. C.-S. Leung, K.-W. Wong, P.-F. Sum, and L.-W. Chan, “A pruning method for the recursive least squared algorithm,” Neural Networks, vol. 14, no. 2, pp. 147–174, 2001.
49. H.-S. Tang, S. Xue, and T. Sato, “${H}_{\infty }$ filtering in neural network training and pruning with application to system identification,” Journal of Computing in Civil Engineering, vol. 21, no. 1, pp. 47–58, 2007.
50. M. E. Ricotti and E. Zio, “Neural network approach to sensitivity and uncertainty analysis,” Reliability Engineering and System Safety, vol. 64, no. 1, pp. 59–71, 1999.
51. D. Sabo and X.-H. Yu, “A new pruning algorithm for neural network dimension analysis,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN '08), pp. 3313–3318, Hong Kong, China, June 2008.
52. H. Chandrasekaran, H.-H. Chen, and M. T. Manry, “Pruning of basis functions in nonlinear approximators,” Neurocomputing, vol. 34, pp. 29–53, 2000.
53. A. Saltelli, K. S. Chan, and E. M. Scott, Sensitivity Analysis, John Wiley & Sons, New York, NY, USA, 2000.
54. F. Gruau, “A learning and pruning algorithm for genetic boolean neural networks,” in Proceedings of the 1st European Symposium on Artificial Neural Networks (ESANN '93), pp. 57–63, Brussels, Belgium, April 1993.
55. D. W. Ruck, S. K. Rogers, and M. Kabrisky, “Feature selection using a multilayer perceptron,” Neural Network Computing, vol. 2, no. 2, pp. 40–48, 1990.
56. G. Tarr, Multilayered feedforward networks for image segmentation [Ph.D. dissertation], Air Force Institute of Technology, Wright-Patterson AFB, Ohio, USA, 1991.
57. L. Ljung, System Identification: Theory for the Users, Prentice-Hall, Englewood Cliffs, NJ, USA, 1987.
58. P. Thomas and G. Bloch, “Initialization of one hidden layer feed-forward neural networks for non-linear system identification,” in Proceedings of the 15th IMACS World Congress on Scientific Computation, Modelling and Applied Mathematics (WC '97), pp. 295–300, Berlin, Germany, August 1997.
59. H. El Haouzi, Approche méthodologique pour l'intégration des systèmes contrôlés par le produit dans un environnement de juste-à-temps: application à l'entreprise TRANE [Ph.D. dissertation], Université Henri Poincaré Nancy 1, Nancy, France, 2008.
60. T. Klein, Le kanban actif pour assurer l'intéropérabilité décisionnelle centralisé/distribué: application à un industriel de l'ameublement [Ph.D. dissertation], Université Henri Poincaré Nancy 1, Nancy, France, 2008.