About this Journal Submit a Manuscript Table of Contents
ISRN Artificial Intelligence
Volume 2012 (2012), Article ID 820364, 12 pages
http://dx.doi.org/10.5402/2012/820364
Research Article

Neural Discriminant Models, Bootstrapping, and Simulation

1Department of Engineering Informatics, Osaka Electro-Communication University, 18-8 Hatsu-chou, Neyagawa, Osaka 572-8530, Japan
2Department of Clinical Research and Development, Otsuka Pharmaceutial Co., Ltd., Osaka, Japan
3Clinical Information Division Data Science Center, EPS Corporation, Japan

Received 8 October 2011; Accepted 30 November 2011

Academic Editor: J. J. Chen

Copyright © 2012 Masaaki Tsujitani et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. C. M. Bishop, Pattern Regression and Machine Learning, Springer, New York, NY, USA, 2006.
  2. J. S. Bridle, “Probabilistic interpretation of feed-forward classification network outputs, with relationships to statistical pattern recognition,” in Neurocomputing: Algorithms, Architectures and Applications, F. F. Soulie and J. Herault, Eds., pp. 227–236, Springer, New York, NY, USA, 1990.
  3. B. Cheng and D. M. Titterington, “Neural networks: a review from statistical perspective,” Statistical Science, vol. 9, pp. 2–54, 1994.
  4. H. Gish, “Maximum likelihood training of neural networks,” in Artificial Intelligence Frontiers in Statistics, D. J. Hand, Ed., pp. 241–255, Chapman & Hall, New York, NY, USA, 1993.
  5. M. D. Richard and R. P. Lippmann, “Neural network classifiers estimate Bayesian a posteriori probabilities,” Neural Computation, vol. 3, pp. 461–483, 1991.
  6. B. D. Ripley, Pattern Recognition and Neural Networks, Cambridge University Press, New York, NY, USA, 1996.
  7. H. White, “Some asymptotic results for learning in single hidden-layer feedforward network models,” Journal of the American Statistical Association, vol. 84, pp. 1003–1013, 1989.
  8. M. Aitkin and R. Foxall, “Statistical modelling of artificial neural networks using the multi-layer perceptron,” Statistics and Computing, vol. 13, no. 3, pp. 227–239, 2003. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  9. B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap, Chapman & Hall, New York, NY, USA, 1993.
  10. H. Akaike, “Information theory and an extension of the maximum likelihood principle,” in Proceedings of the 2nd International Symposium on Information Theory, B. N. Petrov and F. Csaki, Eds., pp. 267–281, Akademia Kaido, Budapest, Hungary, 1973.
  11. G. Schwarz, “Estimating the dimension of a model,” Annals of Statistics, vol. 6, pp. 461–464, 1978.
  12. J. Shao, “Linear model selection by cross-validation,” Journal of the American Statistical Association, vol. 88, pp. 486–494, 1993.
  13. P. Zhang, “Model selection via multifold cross validation,” Annals of Statistics, vol. 21, pp. 299–313, 1993.
  14. T. J. Hastie and R. J. Tibshirani, Generalized Additive Models, Chapman & Hall, New York, NY, USA, 1990.
  15. S. N. Wood, Generalized Additive Models: An Introduction with R, Chapman & Hall, New York, NY, USA, 2006.
  16. N. Cristianini and J. Shawe-Tylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Method, Cambridge University Press, Cambridge, UK, 2000.
  17. Y. J. Lee and S. Y. Huang, “Reduced support vector machines: a statistical theory,” IEEE Transactions on Neural Networks, vol. 18, no. 1, pp. 1–13, 2007. View at Publisher · View at Google Scholar · View at PubMed · View at Scopus
  18. E. Romero and D. Toppo, “Comparing support vector machines and feedforward neural networks with similar hidden-layer weights,” IEEE Transactions on Neural Networks, vol. 18, no. 3, pp. 959–963, 2007. View at Publisher · View at Google Scholar · View at PubMed
  19. Q. Tao, D. Chu, and J. Wang, “Recursive support vector machines for dimensionality reduction,” IEEE Transactions on Neural Networks, vol. 19, no. 1, pp. 189–193, 2008. View at Publisher · View at Google Scholar · View at PubMed · View at Scopus
  20. T. W. Yee and C. J. Wild, “Vector generalized additive models,” Journal of the Royal Statistical Society Series B, vol. 58, pp. 481–493, 1996.
  21. K. I. Funahashi, “On the approximate realization of continuous mappings by neural networks,” Neural Networks, vol. 2, no. 3, pp. 183–192, 1989. View at Scopus
  22. G. Gong, “Cross-validation, the jackknife, and the bootstrap: excess error estimation in forward logistic regression,” Journal of the American Statistical Association, vol. 81, pp. 108–113, 1986.
  23. M. C. Wang, “Re-sampling procedures for reducing bias of error rate estimation in multinomial classification,” Computational Statistics and Data Analysis, vol. 4, no. 1, pp. 15–39, 1986. View at Scopus
  24. U. Anders and O. Korn, “Model selection in neural networks,” Neural Networks, vol. 12, no. 2, pp. 309–323, 1999. View at Publisher · View at Google Scholar · View at Scopus
  25. M. Ishiguro and Y. Sakamoto, “WIC: an estimation-free information criterion,” Research Memorandum of the Institute of Statistical Mathematics, Tokyo, Japan, 1991.
  26. S. Konishi and G. Kitagawa, “Generalised information criteria in model selection,” Biometrika, vol. 83, no. 4, pp. 875–890, 1996. View at Scopus
  27. M. Ishiguro, Y. Sakamoto, and G. Kitagawa, “Bootstrapping log likelihood and EIC, an extension of AIC,” Annals of the Institute of Statistical Mathematics, vol. 49, no. 3, pp. 411–434, 1997. View at Scopus
  28. S. Kullback and R. A. Leibler, “On information and sufficiency,” Annals of Mathematical Statistics, vol. 22, pp. 79–86, 1951.
  29. R. Shibata, “Bootstrap estimate of Kullback-Leibler information for model selection,” Statistica Sinica, vol. 7, no. 2, pp. 375–394, 1997. View at Scopus
  30. J. Shao, “Bootstrap Model Selection,” Journal of the American Statistical Association, vol. 91, no. 434, pp. 655–665, 1996. View at Scopus
  31. R. Tibshirani, “A comparison of some error estimates for neural network models,” Neural Computation, vol. 8, no. 1, pp. 152–163, 1996. View at Scopus
  32. D. Collett, Modeling Binary Data, Chapman & Hall, New York, NY, USA, 2nd edition, 2003.
  33. D. E. Jennings, “Outliers and residual distributions in logistic regression,” Journal of the American Statistical Association, vol. 81, pp. 987–990, 1986.
  34. J. M. Landwehr, D. Pregibon, and A. C. Shoemaker, “Graphical methods for assessing logistic regression models,” Journal of the American Statistical Association, vol. 79, pp. 61–71, 1984.
  35. D. Pregibon, “Logistic regression diagnostics,” Annals of Statistics, vol. 9, pp. 705–724, 1981.
  36. B. Efron, “Estimating the error rate of a prediction rule: improvement on cross-validation,” Journal of the American Statistical Association, vol. 78, pp. 316–331, 1983.
  37. B. Efron, “How biases is the apparent error rate of a prediction rule?” Journal of the American Statistical Association, vol. 81, pp. 461–470, 1986.
  38. S. Eguchi and J. Copas, “A class of logistic-type discriminant functions,” Biometrika, vol. 89, no. 1, pp. 1–22, 2002. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  39. O. Intrator and N. Intrator, “Interpreting neural-network results: a simulation study,” Computational Statistics and Data Analysis, vol. 37, no. 3, pp. 373–393, 2001. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  40. G. Schwarzer, W. Vach, and M. Schumacher, “On the misuses of artificial neural networks for prognostic and diagnostic classification in oncology,” Statistics in Medicine, vol. 19, no. 4, pp. 541–561, 2000. View at Publisher · View at Google Scholar
  41. W. Vach, R. Roßner, and M. Schumacher, “Neural networks and logistic regression: Part II,” Computational Statistics and Data Analysis, vol. 21, no. 6, pp. 683–701, 1996. View at Publisher · View at Google Scholar · View at Scopus
  42. M. Tsujitani and T. Koshimizu, “Neural discriminant analysis,” IEEE Transactions on Neural Networks, vol. 11, no. 6, pp. 1394–1401, 2000. View at Scopus
  43. X. Lin, Smoothing spline analysis of variance for polychotomous response data, Ph.D. thesis, University of Wisconsin, Madison, Wis, USA, 1998.
  44. M. Tsujitani and M. Aoki, “Neural regression model, resampling and diagnosis,” Systems and Computers in Japan, vol. 37, no. 6, pp. 13–20, 2006. View at Publisher · View at Google Scholar · View at Scopus
  45. E. Lesaffre and A. Albert, “Multiple-group logistic regression diagnosis,” Journal of Applied Statistics, vol. 38, pp. 425–440, 1989.
  46. J. M. Chambers and T. J. Hasti, Statistical Models in S, Chapman & Hall, New York, NY, USA, 1992.
  47. T. J. Hastie, R. J. Tibshirani, and J. Friedman, The Elements of Statistical Learning-Data Mining, Inference and Prediction, Springer, New York, NY, USA, 2001.
  48. V. N. Vapnik, Statistical Learning Theory, Wiley, New York, NY, USA, 1998.
  49. T. S. Lim, W. Y. Loh, and Y. S. Shih, “Comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms,” Machine Learning, vol. 40, no. 3, pp. 203–228, 2000. View at Publisher · View at Google Scholar · View at Scopus
  50. Y. Sakurai and Y. Yashiki, “Assessment of land value by the metropolitan area stations,” Weekly Takarajima, no. 572, pp. 24–42, 2002.
  51. R. M. Neal, Bayesian Learning for Neural Networks, Springer, New York, NY, USA, 1996.
  52. C. Gu, Smoothing Spline ANOVA Models, Springer, New York, NY, USA, 2002.
  53. B. Baesens, T. Van Gestel, M. Stepanova, D. Van Den Poel, and J. Vanthienen, “Neural network survival analysis for personal loan data,” Journal of the Operational Research Society, vol. 56, no. 9, pp. 1089–1098, 2005. View at Publisher · View at Google Scholar · View at Scopus
  54. D. R. Mani, J. Drew, A. Betz, and P. Datta, “Statistics and data mining techniques for life-time value modeling,” in Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 94–103, San Diego, Calif, USA, 1999.
  55. W. N. Street, “A neural network model for prognostic prediction,” in Proceedings of the 15th International Conference on Machine Learning, pp. 540–546, Wisconsin, Wis, USA, 1998.