About this Journal Submit a Manuscript Table of Contents
Advances in Artificial Neural Systems
Volume 2013 (2013), Article ID 284570, 17 pages
http://dx.doi.org/10.1155/2013/284570
Research Article

Variance Sensitivity Analysis of Parameters for Pruning of a Multilayer Perceptron: Application to a Sawmill Supply Chain Simulation Model

Centre de Recherche en Automatique de Nancy (CRAN-UMR 7039), Nancy-Université, CNRS, Campus Sciences, BP 70239, 54506 Vandoeuvre les Nancy Cedex, France

Received 6 May 2013; Revised 15 July 2013; Accepted 15 July 2013

Academic Editor: Chao-Ton Su

Copyright © 2013 Philippe Thomas et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. P. Lopez and F. Roubellat, Ordonnancement de la Production, Hermès, Paris, France.
  2. E. Goldratt and J. Cox, The Goal: A Process of Ongoing Improvement, The North River Press, Great Barrington, Mass, USA, 2nd edition, 1992.
  3. A. Thomas and P. Charpentier, “Reducing simulation models for scheduling manufacturing facilities,” European Journal of Operational Research, vol. 161, no. 1, pp. 111–125, 2005. View at Publisher · View at Google Scholar · View at Scopus
  4. E. H. Page, D. M. Nicol, O. Balci et al., “An aggregate production planning framework for the evaluation of volume flexibility,” in Proceedings of the Winter Simulation Conference (WSC '99), pp. 1509–1520, Phoenix, Ariz, USA, December 1999.
  5. S. C. Ward, “Argument for constructively simple models,” Journal of the Operational Research Society, vol. 40, no. 2, pp. 141–153, 1989.
  6. R. J. Brooks and A. M. Tobias, “Simplification in the simulation of manufacturing systems,” International Journal of Production Research, vol. 38, no. 5, pp. 1009–1027, 2000. View at Scopus
  7. L. Chwif, R. J. Paul, and M. R. P. Barretto, “Discrete event simulation model reduction: a causal approach,” Simulation Modelling Practice and Theory, vol. 14, no. 7, pp. 930–944, 2006. View at Publisher · View at Google Scholar · View at Scopus
  8. P. Thomas and A. Thomas, “Expérimentation de la reduction d'un modèle de simulation par réseau de neurones: cas d'une scierie,” in Proceedings of the 7ème Conférence Internationale de Modélisation, Optimisation et Simulation des Systèmes (MOSIM '08), pp. 2030–2038, Paris, France, March-April 2008.
  9. P. Thomas, G. Bloch, F. Sirou, and V. Eustache, “Neural modeling of an induction furnac using robust learning criteria,” Integrated Computer-Aided Engineering, vol. 6, no. 1, pp. 15–26, 1999. View at Scopus
  10. P. Thomas, D. Choffel, and A. Thomas, “Simulation reduction models approach using neural network,” in Proceedings of the 10th International Conference on Computer Modelling and Simulation (EUROSIM '08), Cambridge, UK, April 2008.
  11. P. L. Bartlett, “For valid generalization, the size of the weights is more important than the size of the network,” Neural Information Processing Systems, vol. 9, pp. 134–140, 1997.
  12. G. C. Cawley and N. L. C. Talbot, “Preventing over-fitting during model selection via bayesian regularisation of the hyper-parameters,” Journal of Machine Learning Research, vol. 8, pp. 841–861, 2007. View at Scopus
  13. H. Drucker, “Effect of pruning and early stopping on performance of a boosting ensemble,” Computational Statistics and Data Analysis, vol. 38, no. 4, pp. 393–406, 2002. View at Publisher · View at Google Scholar · View at Scopus
  14. C. M. Bishop, Neural Network for Pattern Recognition, Oxford University Press, Oxford, UK, 1995.
  15. P. Lauret, E. Fock, and T. A. Mara, “A node pruning agorithm based on a fourier amplitude sensitivity test method,” IEEE Transactions on Neural Networks, vol. 17, no. 2, pp. 273–293, 2006. View at Publisher · View at Google Scholar · View at Scopus
  16. R. Chentouf and C. Jutten, “Combining sigmoids and radial basis function in evolutive neural architecture,” in Proceedings of the 4th European Symposium on Artificial Neural Networks (ESANN '96), pp. 129–134, Bruges, Belgium, April 1996.
  17. R. Setiono, “Feedforward neural network construction using cross validation,” Neural Computation, vol. 13, no. 12, pp. 2865–2877, 2001. View at Publisher · View at Google Scholar · View at Scopus
  18. I. Rivals and L. Personnaz, “Neural-network construction and selection in nonlinear modeling,” IEEE Transactions on Neural Networks, vol. 14, no. 4, pp. 804–819, 2003. View at Publisher · View at Google Scholar · View at Scopus
  19. L. Ma and K. Khorasani, “New training strategies for constructive neural networks with application to regression problems,” Neural Networks, vol. 17, no. 4, pp. 589–609, 2004. View at Publisher · View at Google Scholar · View at Scopus
  20. B. Hassibi and D. G. Stork, “Second order derivatives for network pruning: optimal brain surgeon,” in Advances in Neural Information Processing Systems, S. H. Hanson, J. D. Cowan, and C. L. Gilles, Eds., pp. 164–171, Morgan Kaufmann, San Mateo, Calif, USA, 1993.
  21. M. Cottrell, B. Girard, Y. Girard, M. Mangeas, and C. Muller, “Neural modeling for time series: a statistical stepwise method for weight elimination,” IEEE Transactions on Neural Networks, vol. 6, no. 6, pp. 1355–1364, 1995. View at Publisher · View at Google Scholar · View at Scopus
  22. P. Thomas and G. Bloch, “Robust pruning for multilayer perceptrons,” in Proceedings of the 2nd IMACS/IEEE Multiconference on Computational Engineering in Systems Applications (CESA '98), pp. 17–22, Nabeul-Hammamet, Tunisia, April 1998.
  23. J. Xu and D. W. C. Ho, “A new training and pruning algorithm based on node dependence and Jacobian rank deficiency,” Neurocomputing, vol. 70, no. 1–3, pp. 544–558, 2006. View at Publisher · View at Google Scholar · View at Scopus
  24. X. Zeng and D. S. Yeung, “Hidden neuron pruning of multilayer perceptrons using a quantified sensitivity measure,” Neurocomputing, vol. 69, no. 7–9, pp. 825–837, 2006. View at Publisher · View at Google Scholar · View at Scopus
  25. X. Liang, “Removal of hidden neurons in multilayer perceptrons by orthogonal projection and weight crosswise propagation,” Neural Computing and Applications, vol. 16, no. 1, pp. 57–68, 2007. View at Publisher · View at Google Scholar · View at Scopus
  26. E. Romero and J. M. Sopena, “Performing feature selection with multilayer perceptrons,” IEEE Transactions on Neural Networks, vol. 19, no. 3, pp. 431–441, 2008. View at Publisher · View at Google Scholar · View at Scopus
  27. I. T. Jollife, Principal Component Analysis, Springer, New York, NY, USA, 1986.
  28. P. Demartines, Analyse de données par réseaux de neurones auto-organisés [Ph.D. dissertation], Institut National Polytechnique de Grenoble, Valence, France, 1995.
  29. G. Dreyfus, J. M. Martinez, M. Samuelides, et al., Réseaux de Neurones: Méthodologies et Applications, Editions Eyrolles, Paris, France, 2002.
  30. H. Stoppiglia, G. Dreyfus, R. Dubois, and Y. Oussar, “Ranking a random feature for variable and feature selection,” Journal of Machine Learning Research, vol. 3, pp. 1399–1414, 2003.
  31. T. Cibas, F. F. Soulié, P. Gallinari, and S. Raudys, “Variable selection with neural networks,” Neurocomputing, vol. 12, no. 2-3, pp. 223–248, 1996. View at Publisher · View at Google Scholar · View at Scopus
  32. P. Leray and P. Gallinari, “Feature selection with neural networks,” Behaviometrika, vol. 26, no. 1, pp. 145–166, 1999. View at Publisher · View at Google Scholar
  33. G. Castellano and A. M. Fanelli, “Variable selection using neural-network models,” Neurocomputing, vol. 31, no. 1–4, pp. 1–13, 2000. View at Publisher · View at Google Scholar · View at Scopus
  34. S. Gadat and L. Younes, “A stochastic algorithm for feature selection in pattern recognition,” Journal of Machine Learning Research, vol. 8, pp. 509–547, 2007. View at Scopus
  35. R. Setiono and W. K. Leow, “Pruned neural networks for regression,” in Proceedings of the 6th Pacific Rim International Conference on Artificial Intelligence (PRICAI '00), pp. 500–509, Melbourne, Australia, August 2000.
  36. A. P. Engelbrecht, “A new pruning heuristic based on variance analysis of sensitivity information,” IEEE Transactions on Neural Networks, vol. 12, no. 6, pp. 1386–1389, 2001. View at Publisher · View at Google Scholar · View at Scopus
  37. B. P. Zeigler, Theory of Modeling and Simulation, John Wiley & Sons, New York, NY, USA, 1976.
  38. G. Innis and E. Rexstad, “Simulation model simplification techniques,” Simulation, vol. 41, no. 1, pp. 7–15, 1983. View at Publisher · View at Google Scholar · View at Scopus
  39. R. C. Leachman, Preliminary Design and Development of a Corporate Level Production Planning System for the Semi Conductor Industry, Optimization in Industry, Chichester, UK, 1986.
  40. Y.-F. Hung and R. C. Leachman, “Reduced simulation models of wafer fabrication facilities,” International Journal of Production Research, vol. 37, no. 12, pp. 2685–2701, 1999. View at Scopus
  41. S. Hsieh, J.-S. Hwang, and H.-C. Chou, “A petri-net-based structure for AS/RS operation modelling,” International Journal of Production Research, vol. 36, no. 12, pp. 3323–3346, 1998. View at Scopus
  42. G. Cybenko, “Approximation by superpositions of a sigmoidal function,” Mathematics of Control, Signals, and Systems, vol. 2, no. 4, pp. 303–314, 1989. View at Publisher · View at Google Scholar · View at Scopus
  43. K.-I. Funahashi, “On the approximate realization of continuous mappings by neural networks,” Neural Networks, vol. 2, no. 3, pp. 183–192, 1989. View at Scopus
  44. R. Reed, “Pruning algorithms—a survey,” IEEE Transactions on Neural Networks, vol. 4, no. 5, pp. 740–747, 1993. View at Publisher · View at Google Scholar · View at Scopus
  45. C. Jutten and O. Fambon, “Pruning methods: a review,” in Proceedings of the 3rd European Symposium on Artificial Neural Networks (ESANN '95), pp. 129–140, Brussels, Belgium, April 1995.
  46. Y. LeCun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” Advances in Neural Information Processing Systems, vol. 2, pp. 598–605, 1990.
  47. M. Norgaard, System identification and control with neural networks [Ph.D. dissertation], Institute of Automation, Technical University of Denmark, Copenhagen, Denmark, 1996.
  48. C.-S. Leung, K.-W. Wong, P.-F. Sum, and L.-W. Chan, “A pruning method for the recursive least squared algorithm,” Neural Networks, vol. 14, no. 2, pp. 147–174, 2001. View at Publisher · View at Google Scholar · View at Scopus
  49. H.-S. Tang, S. Xue, and T. Sato, “H filtering in neural network training and pruning with application to system identification,” Journal of Computing in Civil Engineering, vol. 21, no. 1, pp. 47–58, 2007. View at Publisher · View at Google Scholar · View at Scopus
  50. M. E. Ricotti and E. Zio, “Neural network approach to sensitivity and uncertainty analysis,” Reliability Engineering and System Safety, vol. 64, no. 1, pp. 59–71, 1999. View at Scopus
  51. D. Sabo and X.-H. Yu, “A new pruning algorithm for neural network dimension analysis,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN '08), pp. 3313–3318, Hong Kong, China, June 2008. View at Publisher · View at Google Scholar · View at Scopus
  52. H. Chandrasekaran, H.-H. Chen, and M. T. Manry, “Pruning of basis functions in nonlinear approximators,” Neurocomputing, vol. 34, pp. 29–53, 2000. View at Publisher · View at Google Scholar · View at Scopus
  53. A. Saltelli, K. S. Chan, and E. M. Scott, Sensitivity Analysis, John Wiley & Sons, New York, NY, USA, 2000.
  54. F. Gruau, “A learning and pruning algorithm for genetic boolean neural networks,” in Proceedings of the 1st European Symposium on Artificial Neural Networks (ESANN '93), pp. 57–63, Brussels, Belgium, April 1993.
  55. D. W. Ruck, S. K. Rogers, and M. Kabrisky, “Feature selection using a multilayer perceptron,” Neural Network Computing, vol. 2, no. 2, pp. 40–48, 1990.
  56. G. Tarr, Multilayered feedforward networks for image segmentation [Ph.D. dissertation], Air Force Institute of Technology, Wright-Patterson AFB, Ohio, USA, 1991.
  57. L. Ljung, System Identification: Theory for the Users, Prentice-Hall, Englewood Cliffs, NJ, USA, 1987.
  58. P. Thomas and G. Bloch, “Initialization of one hidden layer feed-forward neural networks for non-linear system identification,” in Proceedings of the 15th IMACS World Congress on Scientific Computation, Modelling and Applied Mathematics (WC '97), pp. 295–300, Berlin, Germany, August 1997.
  59. H. El Haouzi, Approche méthodologique pour l'intégration des systèmes contrôlés par le produit dans un environnement de juste-à-temps: application à l'entreprise TRANE [Ph.D. dissertation], Université Henri Poincaré Nancy 1, Nancy, France, 2008.
  60. T. Klein, Le kanban actif pour assurer l'intéropérabilité décisionnelle centralisé/distribué: application à un industriel de l'ameublement [Ph.D. dissertation], Université Henri Poincaré Nancy 1, Nancy, France, 2008.