Table of Contents Author Guidelines Submit a Manuscript
Journal of Applied Mathematics
Volume 2013, Article ID 154358, 15 pages
http://dx.doi.org/10.1155/2013/154358
Research Article

An Optimally Generalized Steepest-Descent Algorithm for Solving Ill-Posed Linear Systems

Department of Civil Engineering, National Taiwan University, Taipei 106-17, Taiwan

Received 25 August 2013; Revised 29 October 2013; Accepted 29 October 2013

Academic Editor: Hui-Shen Shen

Copyright © 2013 Chein-Shan Liu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. J. Barzilai and J. M. Borwein, “Two-point step size gradient methods,” Journal of Numerical Analysis, vol. 8, no. 1, pp. 141–148, 1988. View at Publisher · View at Google Scholar · View at Scopus
  2. M. Raydan, “On the barzilai and borwein choice of steplength for the gradient method,” Journal of Numerical Analysis, vol. 13, no. 3, pp. 321–326, 1993. View at Publisher · View at Google Scholar · View at Scopus
  3. M. Raydan, “The Barzilaiai and Borwein gradient method for the large scale unconstrained minimization problem,” Journal on Optimization, vol. 7, no. 1, pp. 26–33, 1997. View at Google Scholar · View at Scopus
  4. A. Friedlander, J. M. Martínez, B. Molina, and M. Raydan, “Gradient method with retards and generalizations,” Journal on Numerical Analysis, vol. 36, no. 1, pp. 275–289, 1999. View at Google Scholar · View at Scopus
  5. M. Raydan and B. F. Svaiter, “Relaxed steepest descent and Cauchy-Barzilai-Borwein method,” Computational Optimization and Applications, vol. 21, no. 2, pp. 155–167, 2002. View at Publisher · View at Google Scholar · View at Scopus
  6. Y. Dai, J. Yuan, and Y.-X. Yuan, “Modified two-point stepsize gradient methods for unconstrained optimization,” Computational Optimization and Applications, vol. 22, no. 1, pp. 103–109, 2002. View at Publisher · View at Google Scholar · View at Scopus
  7. Y.-H. Dai and L.-Z. Liao, “R-linear convergence of the Barzilai and Borwein gradient method,” Journal of Numerical Analysis, vol. 22, no. 1, pp. 1–10, 2002. View at Publisher · View at Google Scholar · View at Scopus
  8. Y.-H. Dai and Y.-X. Yuan, “Alternate minimization gradient method,” Journal of Numerical Analysis, vol. 23, no. 3, pp. 377–393, 2003. View at Publisher · View at Google Scholar · View at Scopus
  9. R. Fletcher, “On the Barzilai-Borwein Method,” in Optimization and Control with Applications, L. Qi, K. Teo, and X. Yang, Eds., vol. 96 of Applied Optimization, pp. 235–256, Springer, New York, NY, USA, 2005. View at Publisher · View at Google Scholar
  10. Y.-X. Yuan, “A new stepsize for the steepest descent method,” Journal of Computational Mathematics, vol. 24, no. 2, pp. 149–156, 2006. View at Google Scholar · View at Scopus
  11. A. Bhaya and E. Kaszkurewicz, Control Perspectives on Numerical Algorithms and Matrix Problems, Advances in Design and Control, Book 10, SIAM, Philadelphia, Pa, USA, 2006.
  12. U. Helmke and J. B. Moore, Optimization and Dynamical Systems, Springer, Berlin, Germany, 1994.
  13. M. K. Gavurin, “Nonlinear functional equations and continuous analogs of iterative methods,” Izvestiya Vysshikh Uchebnykh Zavedenii Matematika, vol. 5, pp. 18–31, 1958. View at Google Scholar
  14. Y. I. Alber, “Continuous processes of the Newton type,” Differential Equations, vol. 7, pp. 1461–1471, 1971. View at Google Scholar
  15. M. W. Hirsch and S. Smale, “On algorithms for solving fx=0,” Communications on Pure and Applied Mathematics, vol. 32, pp. 281–312, 1979. View at Publisher · View at Google Scholar
  16. M. T. Chu, “On the continuous realization of iterative processes,” SIAM Review, vol. 30, no. 3, pp. 375–387, 1988. View at Google Scholar · View at Scopus
  17. I. M. Ortega and W. C. Rheinboldt, Iterative Solutions of Nonlinear Equations in Several Variables, Academic Press, New York, NY, USA, 1970.
  18. A. Bhaya and E. Kaszkurewicz, “Steepest descent with momentum for quadratic functions is a version of the conjugate gradient method,” Neural Networks, vol. 17, no. 1, pp. 65–71, 2004. View at Publisher · View at Google Scholar · View at Scopus
  19. A. Bhaya and E. Kaszkurewicz, “A control-theoretic approach to the design of zero finding numerical methods,” IEEE Transactions on Automatic Control, vol. 52, no. 6, pp. 1014–1026, 2007. View at Publisher · View at Google Scholar · View at Scopus
  20. C.-S. Liu, “A state feedback controller used to solve an ill-posed linear system by a GL(n,) iterative algorithm,” Communications in Numerical Analysis, vol. 2013, Article ID cna-00181, 22 pages, 2013. View at Google Scholar
  21. U. M. Ascher, K. van den Doel, H. Huang, and B. F. Svaiter, “Gradient descent and fast artificial time integration,” Mathematical Modelling and Numerical Analysis, vol. 43, no. 4, pp. 689–708, 2009. View at Publisher · View at Google Scholar
  22. C.-S. Liu, “A revision of relaxed steepest descent method from the dynamics on an invariant manifold,” Computer Modeling in Engineering and Sciences, vol. 80, no. 1, pp. 57–86, 2011. View at Google Scholar · View at Scopus
  23. C.-S. Liu, “Modifications of steepest descent method and conjugate gradient method against noise for ill-posed linear systems,” Communications in Numerical Analysis, vol. 2012, Article ID cna-00115, 24 pages, 2012. View at Publisher · View at Google Scholar
  24. Y.-H. Dai, W. W. Hager, K. Schittkowski, and H. Zhang, “The cyclic Barzilai-Borwein method for unconstrained optimization,” Journal of Numerical Analysis, vol. 26, no. 3, pp. 604–627, 2006. View at Publisher · View at Google Scholar · View at Scopus
  25. G. Frassoldati, L. Zanni, and G. Zanghirati, “New adaptive stepsize selections in gradient methods,” Journal of Industrial and Management Optimization, vol. 4, no. 2, pp. 299–312, 2008. View at Google Scholar · View at Scopus
  26. W. W. Hager, B. A. Mair, and H. Zhang, “An affine-scaling interior-point CBB method for box-constrained optimization,” Mathematical Programming, vol. 119, no. 1, pp. 1–32, 2009. View at Publisher · View at Google Scholar · View at Scopus
  27. S. Bonettini, R. Zanella, and L. Zanni, “A scaled gradient projection method for constrained image deblurring,” Inverse Problems, vol. 25, no. 1, Article ID 015002, 2009. View at Publisher · View at Google Scholar · View at Scopus
  28. G. Yu, L. Qi, and Y. Dai, “On nonmonotone chambolle gradient projection algorithms for total variation image restoration,” Journal of Mathematical Imaging and Vision, vol. 35, no. 2, pp. 143–154, 2009. View at Publisher · View at Google Scholar · View at Scopus
  29. T. Lukić, J. Lindblad, and N. Sladoje, “Regularized image denoising based on spectral gradient optimization,” Inverse Problems, vol. 27, no. 8, Article ID 085010, 2011. View at Publisher · View at Google Scholar · View at Scopus
  30. S. Setzer, G. Steidl, and J. Morgenthaler, “A cyclic projected gradient method,” Computational Optimization and Applications, vol. 54, no. 2, pp. 417–440, 2013. View at Publisher · View at Google Scholar
  31. J. Dongarra and F. Sullivan, “Guest editors introduction to the top 10 algorithms,” Computing in Science & Engineering, vol. 2, no. 1, pp. 22–23, 2000. View at Publisher · View at Google Scholar
  32. Y. Saad, “Krylov subspace methods for solving large unsymmetric linear systems,” Mathematics of Computation, vol. 37, no. 155, pp. 105–126, 1981. View at Google Scholar
  33. R. W. Freund and N. M. Nachtigal, “QMR: a quasi-minimal residual method for non-Hermitian linear systems,” Numerische Mathematik, vol. 60, no. 1, pp. 315–339, 1991. View at Publisher · View at Google Scholar · View at Scopus
  34. J. Van Den Eshof and G. L. G. Sleijpen, “Inexact Krylov subspace methods for linear systems,” SIAM Journal on Matrix Analysis and Applications, vol. 26, no. 1, pp. 125–153, 2005. View at Publisher · View at Google Scholar · View at Scopus
  35. C.-S. Liu, “An optimal multi-vector iterative algorithm in a Krylov subspace for solving the ill-posed linear inverse problems,” Computers Materials & Continua, vol. 33, no. 2, pp. 175–198, 2013. View at Google Scholar
  36. V. Simoncini and D. B. Szyld, “Recent computational developments in Krylov subspace methods for linear systems,” Numerical Linear Algebra with Applications, vol. 14, no. 1, pp. 1–59, 2007. View at Publisher · View at Google Scholar · View at Scopus
  37. Y. Saad and M. H. Schultz, “GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems,” SIAM Journal on Scientific and Statistical Computing, vol. 7, no. 3, pp. 856–869, 1986. View at Publisher · View at Google Scholar
  38. Y. Saad, Iterative Methods for Sparse Linear Systems, SIAM, Philadelphia, Pa, USA, 2nd edition, 2003.
  39. C.-S. Liu, H.-K. Hong, and S. N. Atluri, “Novel algorithms based on the conjugate gradient method for inverting ill-conditioned matrices, and a new regularization method to solve ill-posed linear systems,” Computer Modeling in Engineering and Sciences, vol. 60, no. 3, pp. 279–308, 2010. View at Google Scholar · View at Scopus
  40. C.-S. Liu, “An optimal tri-vector iterative algorithm for solving ill-posed linear inverse problems,” Inverse Problems in Science and Engineering, vol. 21, no. 4, pp. 650–681, 2013. View at Publisher · View at Google Scholar
  41. C.-S. Liu, “A dynamical Tikhonov regularization for solving ill-posed linear algebraic systems,” Acta Applicandae Mathematicae, vol. 123, no. 1, pp. 285–307, 2013. View at Publisher · View at Google Scholar
  42. Y. C. Hon and M. Li, “A discrepancy principle for the source points location in using the MFS for solving the BHCP,” International Journal of Computational Methods, vol. 6, no. 2, pp. 181–197, 2009. View at Publisher · View at Google Scholar · View at Scopus
  43. C.-S. Liu, “The method of fundamental solutions for solving the backward heat conduction problem with conditioning by a new post-conditioner,” Numerical Heat Transfer, Part B, vol. 60, no. 1, pp. 57–72, 2011. View at Publisher · View at Google Scholar · View at Scopus