Table of Contents
Advances in Optical Technologies
Volume 2008, Article ID 546808, 18 pages
http://dx.doi.org/10.1155/2008/546808
Research Article

Spatial, Temporal, and Interchannel Image Data Fusion for Long-Distance Terrestrial Observation Systems

Department of Physical Electronics, School of Electrical Engineering, Faculty of Engineering, Tel Aviv University, Ramat Aviv 69978, Israel

Received 14 May 2007; Revised 24 October 2007; Accepted 16 December 2007

Academic Editor: Michael A. Fiddy

Copyright © 2008 Barak Fishbain et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. M. C. Roggermann and B. Welsh, Imaging Through Turbulence, CRC Press, Boca Raton, Fla, USA, 1996.
  2. L. P. Yaroslavsky, B. Fishbain, G. Shabat, and I. Ideses, “Super-resolution in turbulent videos: making profit from damage,” Optics Letters, vol. 32, no. 20, pp. 3038–3040, 2007. View at Publisher · View at Google Scholar
  3. C. J. Carrano, “Anisoplanatic performance of horizontal-path speckle imaging,” in Advanced Wavefront Control: Methods, Devices, and Applications, vol. 5162 of Proceedings of SPIE, pp. 14–27, San Diego, Calif, USA, August 2003. View at Publisher · View at Google Scholar
  4. C. J. Carrano, “Speckle imaging over horizontal paths,” in High-Resolution Wavefront Control: Methods, Devices, and Applications IV, vol. 4825 of Proceedings of SPIE, pp. 109–120, Seattle, Wash, USA, July 2002. View at Publisher · View at Google Scholar
  5. C. J. Carrano and J. M. Brase, “Adapting high-resolution speckle imaging to moving targets and platforms,” in Airborne Intelligence, Surveillance, Reconnaissance (ISR) Systems and Applications, vol. 5409 of Proceedings of SPIE, pp. 96–105, Orlando, Fla, USA, April 2004. View at Publisher · View at Google Scholar
  6. C. J. Carrano and J. M. Brase, “Horizontal and slant path surveillance with speckle imaging,” in Proceedings of the Technical Conference of Aiforce Maui Optical Station (AMOS '02), p. 499, Maui, Hawaii, USA, September 2002.
  7. T. W. Lawrence, J. P. Fitch, D. M. Goodman, N. A. Massie, R. J. Sherwood, and E. M. Johansson, “Extended-image reconstruction through horizontal path turbulence using bispectral speckle interferometry,” Optical Engineering, vol. 31, no. 3, pp. 627–636, 1992. View at Publisher · View at Google Scholar
  8. R. D. Hudson Jr., Infrared System Engineering, Wiley-Interscience, New York, NY, USA, 1969.
  9. Z. Zhang and R. S. Blum, “Image fusion for a digital camera application,” in Proceedings of the 32nd Asilomar Conference on Signals, Systems & Computers, vol. 1, pp. 603–607, Pacific Grove, Calif, USA, November 1998. View at Publisher · View at Google Scholar
  10. A. M. Achim, C. N. Canagarajah, and D. R. Bull, “Complex wavelet domain image fusion based on fractional lower order moments,” in Proceedings of the 8th International Conference on Information Fusion (FUSION '05), vol. 1, pp. 515–521, Philadelphia, Pa, USA, July 2005. View at Publisher · View at Google Scholar
  11. H. Ghassemian, “Multi-sensor image fusion using multirate filter banks,” in Proceedings of IEEE International Conference on Image Processing (ICIP '01), vol. 1, pp. 846–849, Thessaloniki, Greece, October 2001.
  12. V. Petrovich and C. Xydeas, “Computationally Efficient Pixel-level Image Fusion,” in Proceedings of the International Conference on Data Fusion (EuroFusion '99), pp. 177–184, Stratford-upon-Avon, UK, October 1999.
  13. L. P. Yaroslavsky, B. Fishbain, A. Shteinman, and S. Gepshtein, “Processing and fusion of thermal and video sequences for terrestrial long range observation systems,” in Proceedings of the 7th International Conference on Information Fusion (FUSION '04), vol. 2, pp. 848–855, Stockholm, Sweden, June-July 2004.
  14. Q.-S. Sun, S.-G. Zeng, Y. Liu, P.-A. Heng, and D.-S. Xia, “A new method of feature fusion and its application in image recognition,” Pattern Recognition, vol. 38, no. 12, pp. 2437–2448, 2005. View at Publisher · View at Google Scholar
  15. K. Steinnocher, “Adaptive fusion of multisource raster data applying filter techniques,” in Proceedings of the International Archives of Photogrammetry and Remote Sensing, vol. 32, part 7-4-3W6, pp. 108–115, Valladolid, Spain, June 1999.
  16. N. Nandhakumar and J. K. Aggarwal, “Integrated analysis of thermal and visual images for scene interpretation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 10, no. 4, pp. 469–481, 1988. View at Publisher · View at Google Scholar
  17. D. L. Hall, M. D. McNeese, E. Rotthoff, and T. Shaw, “Improving the fusion process using semantic level whole-brain analysis,” in Proceedings of the MSS National Symposium on Sensors and Data Fusion, Monterey, Calif, USA, May 2005.
  18. J. J. Lewis, R. J. O'Callaghan, S. G. Nikolov, D. R. Bull, and C. N. Canagarajah, “Region-based image fusion using complex wavelets,” in Proceedings of the 7th International Conference on Information Fusion (FUSION '04), vol. 1, pp. 555–562, Stockholm, Sweden, June-July 2004.
  19. T. A. Wilson, S. K. Rogers, and M. Kabrisky, “Perceptual-based image fusion for hyperspectral data,” IEEE Transactions on Geoscience and Remote Sensing, vol. 35, no. 4, pp. 1007–1017, 1997. View at Publisher · View at Google Scholar
  20. P. J. Burt and R. J. Kolczynski, “Enhanced image capture through fusion,” in Proceedings of the 4th IEEE International Conference on Computer Vision, pp. 173–182, Berlin, Germany, May 1993.
  21. S. G. Nikolov, D. R. Bull, C. N. Canagarajah, M. Halliwell, and P. N. T. Wells, “Image fusion using a 3-D wavelet transform,” in Proceedings of the 7th International Conference on Image Processing and Its Applications, vol. 1, pp. 235–239, Manchester, UK, July 1999.
  22. P. Hill, N. Canagarajah, and D. Bull, “Image fusion using complex wavelets,” in Proceedings of the 13th British Machine Vision Conference (BMVC '02), pp. 487–496, Cardiff, UK, 2002.
  23. I. Koren, A. Laine, and F. Taylor, “Image fusion using steerable dyadic wavelet transform,” in Proceedings of the IEEE International Conference on Image Processing (ICIP '95), vol. 3, pp. 232–235, Washington, DC, USA, October 1995.
  24. H. Li, B. S. Manjunath, and S. K. Mitra, “Multisensor image fusion using the wavelet transform,” Graphical Models & Image Processing, vol. 57, no. 3, pp. 235–245, 1995. View at Publisher · View at Google Scholar
  25. E. Lallier and M. Farooq, “A real-time pixel-level based image fusion via adaptive weight averaging,” in Proceedings of the 3rd International Conference on Information Fusion (Fusion '00), vol. 2, pp. WeC3-3–WeC3-10, Paris, France, July 2000.
  26. http://www.eng.tau.ac.il/~barak/RealTimeTurbulenceCompensation.
  27. http://www.eng.tau.ac.il/~yaro/Shtainman/shtainman.htm.
  28. B. M. Welsh and C. S. Gardner, “Performance analysis of adaptive optics systems using slope sensors,” Journal of the Optical Society of America A, vol. 6, no. 12, pp. 1913–1923, 1989. View at Google Scholar
  29. B. L. Ellerbroek, “First-order performance evaluation of adaptive-optics systems for atmospheric-turbulence compensation in extended-field-of-view astronomical telescopes,” Journal of the Optical Society of America A, vol. 11, no. 2, pp. 783–805, 1994. View at Google Scholar
  30. A. Tokovinin, M. Le Louarn, and M. Sarazin, “Isoplanatism in a multiconjugate adaptive optics system,” Journal of the Optical Society of America A, vol. 17, no. 10, pp. 1819–1827, 2000. View at Publisher · View at Google Scholar
  31. M. Lloyd-Hart and N. M. Milton, “Fundamental limits on isoplanatic correction with multiconjugate adaptive optics,” Journal of the Optical Society of America A, vol. 20, no. 10, pp. 1949–1957, 2003. View at Publisher · View at Google Scholar
  32. B. Le Roux, J.-M. Conan, C. Kulcsár, H.-F. Raynaud, L. M. Mugnier, and T. Fusco, “Optimal control law for classical and multiconjugate adaptive optics,” Journal of the Optical Society of America A, vol. 21, no. 7, pp. 1261–1276, 2004. View at Publisher · View at Google Scholar
  33. T. Fusco, J.-M. Conan, L. M. Mugnier, V. Michau, and G. Rousset, “Characterization of adaptive optics point spread function for anisoplanatic imaging. Application to stellar field deconvolution,” Astronomy and Astrophysics Supplement Series, vol. 142, no. 1, pp. 149–156, 2000. View at Publisher · View at Google Scholar
  34. W. M. Farmer, The Atmospheric Filter, vol. 2, JCD Publishing, Bellingham, Wash, USA, 2001.
  35. D. Sadot and N. S. Kopeika, “Imaging through the atmosphere: practical instrumentation-based theory and verification of aerosol modulation transfer function,” Journal of the Optical Society of America A, vol. 10, no. 1, pp. 172–179, 1993. View at Google Scholar
  36. B. Cohen, V. Avrin, M. Belitsky, and I. Dinstein, “Generation of a restored image from a video sequence recorded under turbulence effects,” Optical Engineering, vol. 36, no. 12, pp. 3312–3317, 1997. View at Publisher · View at Google Scholar
  37. B. Ro. Frieden, “Turbulent image reconstruction using object power spectrum information,” Optics Communications, vol. 109, no. 3-4, pp. 227–230, 1994. View at Publisher · View at Google Scholar
  38. H. van-der-Elst and J. J. D. van-Schalkwyk, “Modelling and restoring images distorted by atmospheric turbulence,” in Proceedings of the IEEE South African Symposium on Communications and Signal Processing (COMSIG '94), pp. 162–167, Stellenbosch, South Africa, October 1994. View at Publisher · View at Google Scholar
  39. C. J. Carrano, “Progress in horizontal and slant-path imaging using speckle imaging,” in Optical Engineering at the Lawrence Livermore National Laboratory, vol. 5001 of Proceedings of SPIE, pp. 56–64, San Jose, Calif, USA, January 2003. View at Publisher · View at Google Scholar
  40. D. Fraser, G. Thorpe, and A. Lambert, “Atmospheric turbulence visualization with wide-area motion-blur restoration,” Journal of the Optical Society of America A, vol. 16, no. 7, pp. 1751–1758, 1999. View at Publisher · View at Google Scholar
  41. D. Clyde, I. Scott-Fleming, D. Fraser, and A. Lambert, “Application of optical flow techniques in the restoration of non-uniformly warped images,” in Proceedings of the Digital Image Computing: Techniques and Applications (DICTA '02), pp. 195–200, Melbourne, Australia, January 2002.
  42. I. Scott-Fleming, K. Hege, D. Clyde, D. Fraser, and A. Lambert, “Gradient based optical flow techniques for tracking image motion due to atmospheric turbulence,” in Proceedings of the Signal Recovery and Synthesis Symposium, Optical Society of America, pp. 68–70, Albuquerque, NM, USA, November 2001.
  43. D. H. Frakes, J. W. Monaco, and M. J. T. Smith, “Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '01), vol. 3, pp. 1881–1884, Salt Lake City, Utah, USA, May 2001.
  44. B. Fishbain, L. P. Yaroslavsky, I. A. Ideses, O. Ben-Zvi, and A. Shtern, “Real time stabilization of long range observation system turbulent video,” in Proceedings of the International Society for Optical Engineering, vol. 6496 of Proceedings of SPIE, p. 11 pages, San Jose, Calif, USA, January 2007. View at Publisher · View at Google Scholar
  45. B. Fishbain, I. A. Idess, Sh. Gepstein, and L. P. Yaroslavsky, “Turbulent video enhancement: Image stabilization and super-resolution,” in Proceedings of the 11th Meeting on Optical Engineering and Science, Tel Aviv, Israel, March 2007.
  46. L. P. Yaroslavsky, B. Fishbain, I. A. Ideses, D. Slasky, and Z. Hadas, “Simple methods for real-time stabilization of turbulent video,” in Proceedings of the ICO Topical Meeting on Optoinformatics/Information Photonics (ITMO '06), pp. 138–140, St Petersburg, Russia, October 2006.
  47. S. Gepshtein, A. Shtainman, B. Fishbain, and L. P. Yaroslavsky, “Restoration of atmospheric turbulent video containing real motion using elastic image registration,” in Proceedings of the European Signal Processing Conference (EUSIPCO '04), pp. 477–480, John Platt, Vienna, Austria, September 2004.
  48. L. P. Yaroslavsky, Digital Holography and Digital Image Processing, Kluwer Academic, Boston, Mass, USA, 2003.
  49. B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Real-time stabilization of long range observation system turbulent video,” Journal of Real-Time Image Processing, vol. 2, no. 1, pp. 11–22, 2007. View at Publisher · View at Google Scholar
  50. S.-C. S. Cheung and C. Kamath, “Robust techniques for background subtraction in urban traffic video,” in Visual Communications and Image Processing 2004, vol. 5308 of Proceedings of SPIE, pp. 881–892, San Jose, Calif, USA, January 2004. View at Publisher · View at Google Scholar
  51. T. Amiaz and N. Kiryati, “Piecewise-smooth dense optical flow via level sets,” International Journal of Computer Vision, vol. 68, no. 2, pp. 111–124, 2006. View at Publisher · View at Google Scholar
  52. R. Ben-Ari and N. Sochen, “A general framework for regularization in PDE based computation of optical flow via embedded maps and minimal surfaces,” in roceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '06), New-York, NY, USA, June 2006.
  53. A. Bruhn, J. Weickert, T. Kohlberger, and C. Schnörr, Scale Space and PDE Methods in Computer Vision, Lecture Notes in Computer Science, Springer, Berlin, Germany, 2005.
  54. T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High accuracy optical flow estimation based on theory for wrapping,” in Proceedings of the 8th European Conference on Computer Vision (ECCV '04), vol. 4, pp. 25–36, Prague, Czech Republic, May 2004.
  55. L. J. Barron, D. J. Fleet, and S. S. Beachemin, “Performance of optical flow techniques,” International Journal of Computer Vision, vol. 12, no. 1, pp. 43–77, 1994. View at Publisher · View at Google Scholar
  56. B. K. P. Horn and B. Schunck, “Determining optical flow,” Artificial Intelligence, vol. 17, no. 1–3, pp. 185–203, 1981. View at Publisher · View at Google Scholar
  57. B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI '81), pp. 674–679, Vancouver, BC, Canada, August 1981.
  58. S. Periaswamy and H. Farid, “Elastic registration in the presence of intensity variations,” IEEE Transactions on Medical Imaging, vol. 22, no. 7, pp. 865–874, 2003. View at Publisher · View at Google Scholar
  59. S. Periaswamy and H. Farid, “Differential elastic image registration,” Tech. Rep. TR2001-413, Dartmouth College, Computer Science, Hanover, NH, USA, 2001. View at Google Scholar
  60. M. I. Charnotskii, “Imaging in turbulence beyond diffraction limits,” in Adaptive Optical Systems and Applications, vol. 2534 of Proceedings of SPIE, pp. 289–297, San Diego, Calif, USA, July 1995. View at Publisher · View at Google Scholar
  61. A. Lambert, D. Fraser, M. R. S. Jahromi, and B. R. Hunt, “Super-resolution in image restoration of wide area images viewed through atmospheric turbulence,” in Image Reconstruction from Incomplete Data II, vol. 4792 of Proceedings of SPIE, pp. 35–43, Seattle, Wash, USA, July 2002. View at Publisher · View at Google Scholar
  62. M. I. Charnotskii, V. A. Myakinin, and V. U. Zavorotnyy, “Observation of superresolution in nonisoplanatic imaging through turbulence,” Journal of the Optical Society of America A, vol. 7, no. 8, pp. 1345–1350, 1990. View at Google Scholar
  63. A. Lambert and D. Fraser, “Superresolution in imagery arising from observation through anisoplanatic distortion,” in Image Reconstruction from Incomplete Data III, vol. 5562 of Proceedings of SPIE, pp. 65–75, Denver, Colo, USA, August 2004. View at Publisher · View at Google Scholar
  64. S. Srinivasan and R. Chappella, “Image sequence stabilization, mosaicking and superresolution,” in Video and Image Processing Handbook, A. C. Bovic, Ed., pp. 259–268, Academic Press, Dallas, Tex, USA, 2000, chapter 3.13. View at Google Scholar
  65. M. Irani and S. Peleg, “Improving resolution by image registration,” CVGIP: Graphical Models & Image Processing, vol. 53, no. 3, pp. 231–239, 1991. View at Publisher · View at Google Scholar
  66. N. Goldberg, A. Feuer, and G. C. Goodwin, “Super-resolution reconstruction using spatio-temporal filtering,” Journal of Visual Communication and Image Representation, vol. 14, no. 4, pp. 508–525, 2003. View at Publisher · View at Google Scholar
  67. S. Farsiu, M. Elad, and P. Milanfar, “Multiframe demosaicing and super-resolution of color images,” IEEE Transactions on Image Processing, vol. 15, no. 1, pp. 141–159, 2006. View at Publisher · View at Google Scholar
  68. B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Real-time turbulent video perfecting by image stabilization and super-resolution,” in Proceedings of the 7th IASTED International Conference on Visualization, Imaging, and Image Processing (VIIP '07), Palma de Mallorca, Spain, August 2007.
  69. L. P. Yaroslavsky, “Fast discrete sinc-interpolation: a gold standard for image resampling,” in Advances in Signal Transforms: Theory and Applications, J. Astola and L. P. Yaroslavsky, Eds., EURASIP Book Series on Signal Processing and Communications, Hindawi, New York, NY, USA, 2007. View at Google Scholar
  70. F. Crete, T. Dolmiere, P. Ladref, and M. Nicolas, “The blur effect: perception and estimation with a new no-reference perceptual blur metric,” in Human Vision and Electronic Imaging XII, vol. 6492 of Proceedings of SPIE, p. 11 pages, San Jose, Calif, USA, January 2007. View at Publisher · View at Google Scholar
  71. G. C. Holst, Testing and Evaluation of Infrared Imaging Systems, SPIE Press, Bellingham, Wash, USA, 2nd edition, 1998.
  72. G. C. Holst, CCD Arrays, Cameras and Displays, SPIE Press, Bellingham, Wash, USA, 2nd edition, 2001.
  73. L. P. Yaroslavsky, “Local adaptive filtering in transform domain for image restoration, enhancement and target location,” in Proceedings of the 6th International Workshop on Digital Image Processing and Computer Graphics: Applications in Humanities and Natural Sciences , vol. 3346 of Proceedings of SPIE, pp. 2–17, Vienna, Austria, October 1997. View at Publisher · View at Google Scholar
  74. L. P. Yaroslavsky, “Space variant and adaptive transform domain image and video restoration methods,” in Advances in Signal Transforms: Theory and Applications, J. Astola and L. P. Yaroslavsky, Eds., Hindawi, New York, NY, USA, 2007. View at Google Scholar
  75. L. P. Yaroslavsky and H. J. Caulfield, “Deconvolution of multiple images of the same object,” Applied Optics, vol. 33, no. 11, pp. 2157–2162, 1994. View at Google Scholar
  76. L. P. Yaroslavsky, “Advanced image processing Lab—a tutorial,” in Proceedings of the European Signal Processing Conference (EUSIPCO '00), Tampere, Finland, September 2000.
  77. M. C. Dudzik, Ed., The Infrared & EO Systems Handbook, Electro-Optical Systems Design, Analysis, and Testing, vol 4, vol. 2 of Atmospheric Propagation of Radiation, SPIE Optical Engineering Press, Bellingham, Wash, USA, 1993.
  78. R. L. Lagendijk, P. M. B. van-Roosmalen, and J. Biemond, “Video enhancement and restoration,” in Handbook of Image and Video Processing, Al. Bovik, Ed., pp. 227–241, Academic Press, Canada, 2000. View at Google Scholar
  79. L. P. Yaroslavsky and M. Eden, Fundamentals of Digital Optics, Birkhauser, Boston, Mass, USA, 1996.