About this Journal Submit a Manuscript Table of Contents
ISRN Artificial Intelligence
Volume 2012 (2012), Article ID 486361, 9 pages
http://dx.doi.org/10.5402/2012/486361
Research Article

An Advanced Conjugate Gradient Training Algorithm Based on a Modified Secant Equation

1Department of Mathematics, University of Patras, 26500 Patras, Greece
2Educational Software Development Laboratory, Department of Mathematics, University of Patras, 26500 Patras, Greece

Received 5 August 2011; Accepted 4 September 2011

Academic Editors: T. Kurita and Z. Liu

Copyright © 2012 Ioannis E. Livieris and Panagiotis Pintelas. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Conjugate gradient methods constitute excellent neural network training methods characterized by their simplicity, numerical efficiency, and their very low memory requirements. In this paper, we propose a conjugate gradient neural network training algorithm which guarantees sufficient descent using any line search, avoiding thereby the usually inefficient restarts. Moreover, it achieves a high-order accuracy in approximating the second-order curvature information of the error surface by utilizing the modified secant condition proposed by Li et al. (2007). Under mild conditions, we establish that the proposed method is globally convergent for general functions under the strong Wolfe conditions. Experimental results provide evidence that our proposed method is preferable and in general superior to the classical conjugate gradient methods and has a potential to significantly enhance the computational efficiency and robustness of the training process.