EURASIP Journal on Advances in Signal Processing
Volume 2008 (2008), Article ID 429128, 12 pages
doi:10.1155/2008/429128
Research Article

Iterative Estimation Algorithms Using Conjugate Function Lower Bound and Minorization-Maximization with Applications in Image Denoising

Guang Deng1 and Wai-Yin Ng2

1Department of Electronic Engineering, La Trobe University, Bundoora, Victoria 3086, Australia
2Department of Information Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong

Received 19 September 2007; Revised 3 January 2008; Accepted 11 February 2008

Academic Editor: Hubert Cardot

Copyright © 2008 Guang Deng and Wai-Yin Ng. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. S. M. Kay, Fundamentals of Statistical Signal Processing Estimation Theory, Prentice Hall, Englewood Cliffs, NJ, USA, 1993.
  2. P. J. Huber, Robust Statistics, John Wiley & Sons, New York, NY, USA, 1981.
  3. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, New York, NY, USA, 2001.
  4. C. M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, London, UK, 1995.
  5. B. A. Olshausen and D. J. Field, “Emergence of simple-cell receptive field properties by learning a sparse code for natural images,” Nature, vol. 381, no. 6583, pp. 607–609, 1996.
  6. M. A. T. Figueiredo, “Adaptive sparseness for supervised learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1150–1159, 2003.
  7. J. Fan and R. Li, “Variable selection via nonconcave penalized likelihood and its oracle properties,” Journal of the American Statistical Association, vol. 96, no. 456, pp. 1348–1360, 2001.
  8. R. Tibshirani, “Regression shrinkage and selection via lasso,” Journal of the Royal Statistical Society. Series B, vol. 57, pp. 257–288, 1995.
  9. A. Antoniadis and J. Fan, “Regularization of wavelet approximations,” Journal of the American Statistical Association, vol. 96, no. 455, pp. 939–967, 2001.
  10. D. Donoho and I. Johnstone, “Adapting to unknown smoothness via wavelet shrinkage,” Journal of the American Statistical Association, vol. 90, no. 432, pp. 1200–1224, 1995.
  11. J. M. Bioucas-Dias, “Bayesian wavelet-based image deconvolution: a GEM algorithm exploiting a class of heavy-tailed priors,” IEEE Transactions on Image Processing, vol. 15, no. 4, pp. 937–951, 2006.
  12. D. F. Andrews and C. L. Mallows, “Scale mixtures of normal distributions,” Journal of the Royal Statistical Society. Series B, vol. 36, pp. 99–102, 1974.
  13. W. H. Rogers and J. W. Tukey, “Understanding some long-tailed distributions,” Statistica Neerlandica, vol. 26, pp. 211–226, 1962.
  14. K. L. Lange and J. S. Sinsheimer, “Normal/independent distributions and their applications in robust regression,” Journal of Computational and Graphical Statistics, vol. 2, no. 2, pp. 175–198, 1993.
  15. L. Chuanhai, “Bayesian robust multivariate linear regression with incomplete data,” Journal of the American Statistical Association, vol. 91, no. 435, pp. 1219–1227, 1996.
  16. J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, “Image denoising using scale mixtures of Gaussians in the wavelet domain,” IEEE Transactions on Image Processing, vol. 12, no. 11, pp. 1338–1351, 2003.
  17. D. P. Wipf and B. D. Rao, “Sparse Bayesian learning for basis selection,” IEEE Transactions on Signal Processing, vol. 52, no. 8, pp. 2153–2164, 2004.
  18. F. Girosi, “Models of noise and robust estimates,” Massachusetts Institute of Technology, Cambridge, Mass, USA, 1991.
  19. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul, “An introduction to variational methods for graphical models,” in Learning in Graphical Models, M. I. Jordan, Ed., Kluwer Academic Publishers, Dordrecht, The Netherlands, 1999.
  20. T. S. Jaakkola, “Tutorial on variational approximation methods,” in Advanced Mean Field Methods: Theory and Practice, D. Saad and M. Opper, Eds., MIT Press, Cambridge, Mass, USA, 2000.
  21. D. R. Hunter and K. Lange, “A tutorial on MM algorithms,” American Statistician, vol. 58, no. 1, pp. 30–37, 2004.
  22. M. J. Beal, Variational algorithms for approximate Bayesian inference, Ph.D. dissertation, The Gatsby Computational Neuroscience Unit, University College of London, London, UK, May 2003.
  23. S. J. Roberts and W. D. Penny, “Variational Bayes for generalized autoregressive models,” IEEE Transactions on Signal Processing, vol. 50, no. 9, pp. 2245–2257, 2002.
  24. M. Girolami, “A variational method for learning sparse and overcomplete representations,” Neural Computation, vol. 13, no. 11, pp. 2517–2532, 2001.
  25. M. Welling, G. E. Hinton, and S. Osindero, “Learning sparse topographic representations with products of student-t distributions,” in Proceedings of the 15th Conference on Neural Information Processing Systems (NIPS '02), G. Tesauro, D. S. Touretzky, and T. K. Leen, Eds., pp. 1359–1366, MIT Press, Vancouver, BC, Canada, December 2002.
  26. A. C. Likas and N. P. Galatsanos, “A variational approach for Bayesian blind image deconvolution,” IEEE Transactions on Signal Processing, vol. 52, no. 8, pp. 2222–2233, 2004.
  27. D. R. Hunter and R. Li, “Variable selection using MM algorithms,” Annals of Statistics, vol. 33, no. 4, pp. 1617–1642, 2005.
  28. D. R. Hunter and K. Lange, “Quantile regression via an MM algorithm,” Journal of Computational & Graphical Statistics, vol. 9, pp. 60–77, 2000.
  29. Z. Zhang, J. T. Kwok, and D.-Y. Yeung, “Surrogate maximization/minimization algorithms for AdaBoost and the logistic regression model,” in Proceedings of the 21st International Conference on Machine Learning (ICML '04), pp. 927–934, Banff, Canada, July 2004.
  30. D. Geman and G. Reynolds, “Constrained restoration and the recovery of discontinuities,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 3, pp. 367–383, 1992.
  31. D. Geman and C. Yang, “Nonlinear image recovery with half-quadratic regularization,” IEEE Transactions on Image Processing, vol. 4, no. 7, pp. 932–946, 1995.
  32. M. A. T. Figueiredo and R. D. Nowak, “A bound optimization approach to wavelet-based image deconvolution,” in Proceedings of the International Conference on Image Processing (ICIP '05), vol. 2, pp. 782–785, Genova, Italy, September 2005.
  33. J. Bioucas-Dias, M. Figueiredo, and J. Oliveira, “Total variation image deconvolution: a majorization-minimization approach,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '06), vol. 2, Toulouse, France, May 2006.
  34. G. Deng and W.-Y. Ng, “A minorization-maximization algorithm for maximum a posteriori signal estimation,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '06), vol. 2, pp. 617–620, Toulouse, France, May 2006.
  35. K. Lange and J. Fessler, “Globally convergent algorithms for maximuma posteriori transmission tomography,” IEEE Transactions on Image Processing, vol. 4, no. 10, pp. 1430–1438, 1995.
  36. A. de Pierro, “A modified expectation maximization algorithm for penalized likelihood estimation in emission tomography,” IEEE Transactions on Medical Imaging, vol. 14, no. 1, pp. 132–137, 1995.
  37. P. Charbonnier, L. Blanc-Féraud, G. Aubert, and M. Barlaud, “Deterministic edge-preserving regularization in computed imaging,” IEEE Transactions on Image Processing, vol. 6, no. 2, pp. 298–311, 1997.
  38. H. Erdogan and J. A. Fessler, “Monotonic algorithms for transmission tomography,” IEEE Transactions on Medical Imaging, vol. 18, no. 9, pp. 801–814, 1999.
  39. L. Şendur and I. W. Selesnick, “Bivariate shrinkage with local variance estimation,” IEEE Signal Processing Letters, vol. 9, no. 12, pp. 438–441, 2002.
  40. X. Li and M. T. Orchard, “Spatially adaptive image denoising under overcomplete expansion,” in Proceedings of IEEE International Conference on Image Processing (ICIP '00), vol. 3, pp. 300–303, Vancouver, BC, Canada, September 2000.
  41. S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, Cambridge, UK, 2004.
  42. D. R. Hunter, K. Lange, and I. Yang, “Optimization transfer using surrogate objective functions,” Journal of Computational and Graphical Statistics, vol. 9, no. 1, pp. 1–20, 2000.
  43. M. Jamshidian, “Adaptive robust regression by using a nonlinear regression program,” Journal of Statistical Software, vol. 4, pp. 1–25, 1999.
  44. G. Deng, “Generalized wiener estimation algorithms based on a family of heavy-tail distributions,” in Proceedings of IEEE International Conference on Image Processing (ICIP '05), vol. 1, pp. 457–460, Genova, Italy, September 2005.
  45. A. Gelman, H. B. Carlin, H. S. Stern, and D. B. Rubin, Bayesian Data Analysis, Chapman & Hall/CRC, Boca Raton, Fla, USA, 2004.
  46. G. Deng, “Signal estimation using multiple-wavelet representations and Gaussian models,” in Proceedings of IEEE International Conference on Image Processing (ICIP '05), vol. 1, pp. 453–456, Genova, Italy, September 2005.
  47. L. Şendur and I. W. Selesnick, “Bivariate shrinkage functions for wavelet-based denoising exploiting interscale dependency,” IEEE Transactions on Signal Processing, vol. 50, no. 11, pp. 2744–2756, 2002.
  48. S. G. Chang, B. Yu, and M. Vetterli, “Adaptive wavelet thresholding for image denoising and compression,” IEEE Transactions on Image Processing, vol. 9, no. 9, pp. 1532–1546, 2000.
  49. M. K. Mihcak, I. Kozintsev, K. Ramchandram, and P. Moulin, “Low-complexity image denoising based on statistical modelling of wavelet coefficients,” IEEE Signal Processing Letters, vol. 6, no. 12, pp. 300–303, 1999.
  50. M. Elad, “Why simple shrinkage is still relevant for redundant representations?,” IEEE Transactions on Information Theory, vol. 52, no. 12, pp. 5559–5569, 2006.
  51. S. S. Chen, Basis pursuit, Ph.D. dissertation, Department of Statistics, Stanford University, Stanford, Calif, USA, November 1995.