About this Journal Submit a Manuscript Table of Contents
Mathematical Problems in Engineering
Volume 2012 (2012), Article ID 856523, 17 pages
http://dx.doi.org/10.1155/2012/856523
Research Article

Key Issues in Modeling of Complex 3D Structures from Video Sequences

1College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China
2College of Information Engineering, Zhejiang University of Technology, Hangzhou 310023, China
3Department of Mathematics, University of Salerno, Via Ponte Don Melillo, 84084 Fisciano, Italy

Received 1 July 2011; Accepted 22 August 2011

Academic Editor: Gani Aldashev

Copyright © 2012 Shengyong Chen et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. H. Kuffar and K. Takaya, “Depth measurement and 3D metric reconstruction from two uncalibrated stereo images,” in Proceedings of the Canadian Conference on Electrical and Computer Engineering (CCECD '07), pp. 1460–1463, April 2007. View at Publisher · View at Google Scholar · View at Scopus
  2. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2nd edition, 2004.
  3. O. Faugeras and Q. T. Luong, The Geometry of Multiple Images, MIT Press, Cambridge, Mass, USA, 2001.
  4. R. I. Hartley and P. Sturm, “Triangulation,” in Proceedings of the American Image Understanding Workshop, pp. 957–966, 1994.
  5. P. F. McLauchlan and D. W. Murray, “Unifying framework for structure and motion recovery from image sequences,” in Proceedings of the 5th International Conference on Computer Vision (ICCV '95), pp. 314–320, June 1995. View at Scopus
  6. M. Pollefeys, L. Van Gool, M. Vergauwen et al., “Visual modeling with a hand-held camera,” International Journal of Computer Vision, vol. 59, no. 3, pp. 207–232, 2004. View at Publisher · View at Google Scholar · View at Scopus
  7. S. Avidan and A. Shashua, “Trajectory triangulation: 3D reconstruction of moving points from a monocular image sequence,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 4, pp. 348–357, 2000. View at Publisher · View at Google Scholar · View at Scopus
  8. N. Molton and M. Brady, “Practical structure and motion from stereo when motion is unconstrained,” International Journal of Computer Vision, vol. 39, no. 1, pp. 5–23, 2000. View at Publisher · View at Google Scholar · View at Scopus
  9. A. Marugame, J. Katto, and M. Ohta, “Structure recovery with multiple cameras from scaled orthographic and perspective views,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 7, pp. 628–633, 1999. View at Publisher · View at Google Scholar · View at Scopus
  10. A. Shashua and N. Navab, “Relative affine structure: canonical model for 3D from 2D geometry and applications,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 9, pp. 873–883, 1996. View at Scopus
  11. F. Kahl and A. Heyden, “Affine structure and motion from points, lines and conics,” International Journal of Computer Vision, vol. 33, no. 3, pp. 163–180, 1999. View at Publisher · View at Google Scholar · View at Scopus
  12. A. W. Fitzgibbon and A. Zisserman, “Automatic camera recovery for closed or open image sequences,” in Proceedings of the ECCV, pp. 311–326, 1998.
  13. F. Schaffalitzky and A. Zisserman, “Multi-view matching for unordered image sets, or “How do I organize my holiday snaps?”,” in Proceedings of the IEEE Conference on Computer Vision, vol. 1, pp. 414–431, 2002.
  14. N. Snavely, S. M. Seitz, and R. Szeliski, “Photo tourism: exploring photo collections in 3D,” in Proceedings of the International Conference on Computer Graphics and Interactive Technologies, pp. 835–846, 2006.
  15. R. Hartley, “Euclidean reconstruction from uncalibrated views,” in Applications of Invariance in Computer Vision, J. L. Mundy, A. Zisserman, and D. Forsyth, Eds., Lecture Notes in Computer, 1994.
  16. A. Bartoli and P. Sturm, “Constrained structure and motion from multiple uncalibrated views of a piecewise planar scene,” International Journal of Computer Vision, vol. 52, no. 1, pp. 45–64, 2003. View at Publisher · View at Google Scholar · View at Scopus
  17. R. Szeliski and S. B. Kang, “Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares,” Journal of Visual Communication and Image Representation, vol. 5, no. 1, pp. 10–28, 1994. View at Publisher · View at Google Scholar · View at Scopus
  18. P. M. Q. Aguiar and J. M. F. Moura, “A fast algorithm for rigid structure from image sequences,” in Proceedings of the IEEE International Conference on Image Processing (ICIP '99), vol. 3, pp. 125–129, Kobe, Japan, 1999.
  19. A. Hilton, “Scene modelling from sparse 3D data,” Image and Vision Computing, vol. 23, no. 10, pp. 900–920, 2005. View at Publisher · View at Google Scholar · View at Scopus
  20. P. Parodi and G. Piccioli, “3D shape reconstruction by using vanishing points,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 2, pp. 211–217, 1996. View at Scopus
  21. M. Lhuillier, “Toward flexible 3D modeling using a catadioptric camera,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '07), pp. 1560–1567, June 2007. View at Publisher · View at Google Scholar · View at Scopus
  22. A. Bartoli and S. I. Olsen, “A batch algorithm for implicit non-rigid shape and motion recovery,” in Proceedings of the International Conference on Dynamical Vision, 2006.
  23. D. Liebowitz and S. Carlsson, “Uncalibrated motion capture exploiting articulated structure constraints,” International Journal of Computer Vision, vol. 51, no. 3, pp. 171–187, 2003. View at Publisher · View at Google Scholar · View at Scopus
  24. G. Wang and Q. M. J. Wu, “Stratification approach for 3-D euclidean reconstruction of nonrigid objects from uncalibrated image sequences,” IEEE Transactions on Systems, Man, and Cybernetics, Part B, vol. 38, no. 1, pp. 90–101, 2008. View at Publisher · View at Google Scholar · View at Scopus
  25. J. Taylor, A. D. Jepson, and K. N. Kutulakos, “Non-rigid structure from locally-rigid motion,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '10), pp. 2761–2768, June 2010. View at Publisher · View at Google Scholar · View at Scopus
  26. S. Zhu, L. Zhang, and B. M. Smith, “Model evolution: an incremental approach to non-rigid structure from motion,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '10), pp. 1165–1172, June 2010. View at Publisher · View at Google Scholar · View at Scopus
  27. M. Farenzena and A. Fusiello, “Stabilizing 3D modeling with geometric constraints propagation,” Computer Vision and Image Understanding, vol. 113, no. 11, pp. 1147–1157, 2009. View at Publisher · View at Google Scholar · View at Scopus
  28. R. Gong and G. Xu, “3D structure from a single calibrated view using distance constraints,” IEICE Transactions on Information and Systems, vol. 87, no. 6, pp. 1527–1536, 2004. View at Scopus
  29. M. Marques and J. Costeira, “Estimating 3D shape from degenerate sequences with missing data,” Computer Vision and Image Understanding, vol. 113, no. 2, pp. 261–272, 2009. View at Publisher · View at Google Scholar · View at Scopus
  30. T. Brodsky, C. Fermuller, and Y. Aloimonos, “Structure from motion: beyond the epipolar constraint,” International Journal of Computer Vision, vol. 37, no. 3, pp. 231–258, 2000. View at Publisher · View at Google Scholar · View at Scopus
  31. A. Criminisi, I. Reid, and A. Zisserman, “Single view metrology,” International Journal of Computer Vision, vol. 40, no. 2, pp. 123–148, 2000. View at Publisher · View at Google Scholar · View at Scopus
  32. H. Joachim, “Direct estimation of structure and motion from multiple zframes,” MIT AI Lab. Memo 1190, Massachusetts Institute of Technology, Mass, USA, 190.
  33. J. Oliensis, “Critique of structure-from-motion algorithms,” Computer Vision and Image Understanding, vol. 80, no. 2, pp. 172–214, 2000. View at Publisher · View at Google Scholar · View at Scopus
  34. T. Rodriguez, P. Sturm, P. Gargallo et al., “Photorealistic 3D reconstruction from handheld cameras,” Machine Vision and Applications, vol. 16, no. 4, pp. 246–257, 2005. View at Publisher · View at Google Scholar · View at Scopus
  35. S. N. Sinha, D. Steedly, R. Szeliski, M. Agrawala, and M. Pollefeys, “Interactive 3D architectural modeling from unordered photo collections,” ACM Transactions on Graphics, vol. 27, no. 5, article 159, 2008. View at Publisher · View at Google Scholar · View at Scopus
  36. Y. Furukawa, B. Curless, S. M. Seitz, and R. Szeliski, “Reconstructing building interiors from images,” in Proceedings of the International Conference on Computer Vision, pp. 80–87, 2009.
  37. S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski, “Building Rome in a day,” in Proceedings of the 12th International Conference on Computer Vision (ICCV '09), pp. 72–79, Kyoto, Japan, October 2009. View at Publisher · View at Google Scholar · View at Scopus
  38. Y. Furukawa, B. Curless, S. M. Seitz, and R. Szeliski, “Towards internet-scale multi-view stereo,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '10), pp. 1434–1441, San Francisco, Calif, USA, June 2010. View at Publisher · View at Google Scholar · View at Scopus
  39. Y. T. Zheng, M. Zhao, Y. Song et al., “Tour the World: building a web-scale landmark recognition engine,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR '09), pp. 1085–1092, June 2009. View at Publisher · View at Google Scholar · View at Scopus
  40. M. Pollefeys, et al., “Detailed real-time urban 3D reconstruction from video,” International Journal of Computer Vision, vol. 78, no. 2-3, pp. 143–167, 2008. View at Publisher · View at Google Scholar · View at Scopus
  41. O. Faugeras, L. Robert, S. Laveau et al., “3-D reconstruction of urban scenes from image sequences,” Computer Vision and Image Understanding, vol. 69, no. 3, pp. 292–309, 1998. View at Scopus
  42. N. Cornelis, B. Leibe, K. Cornelis, and L. Van Gool, “3D urban scene modeling integrating recognition and reconstruction,” International Journal of Computer Vision, vol. 78, no. 2-3, pp. 121–141, 2008. View at Publisher · View at Google Scholar · View at Scopus
  43. C. Collewet and F. Chaumette, “Visual servoing based on structure from controlled motion or on robust statistics,” IEEE Transactions on Robotics, vol. 24, no. 2, pp. 318–330, 2008. View at Publisher · View at Google Scholar · View at Scopus
  44. M. Marengoni, A. Hanson, S. Zilberstein, and E. Riseman, “Decision making and uncertainty management in a 3D reconstruction system,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 852–858, 2003. View at Publisher · View at Google Scholar · View at Scopus
  45. T. Tung, F. Schmitt, and T. Matsuyama, “Topology matching for 3D video compression,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '07), pp. 2719–2726, June 2007. View at Publisher · View at Google Scholar · View at Scopus
  46. D. Fidaleo and G. Medioni, “Model-assisted 3D face reconstruction from video,” in Proceedings of the 3rd International Workshop on Analysis and Modeling of Faces and Gestures (AMFG '07), vol. 4778 of Lecture Notes in Computer Science, pp. 124–138, 2007.
  47. U. Park and A. Jain, “3D model-based face recognition in video,” in Proceedings of the Proceedings International Conference on Advances in Biometrics (ICB '07), vol. 4642 of Lecture Notes in Computer Science, pp. 1085–1094, 2007.
  48. V. Nedovic, A. W. M. Smeulders, A. Redert, and J. M. Geusebroek, “Stages as models of scene geometry,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1673–1687, 2010. View at Publisher · View at Google Scholar · View at Scopus
  49. B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679, 1981.
  50. C. Tomasi and T. Kanade, “Detection and tracking of point features,” Tech. Rep. CMU-91-132, CMU, 1991.
  51. J. Shi and C. Tomasi, “Good features to track,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 593–600, June 1994. View at Scopus
  52. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004. View at Publisher · View at Google Scholar · View at Scopus
  53. N. Snavely, I. Simon, M. Goesele, R. Szeliski, and S. M. Seitz, “Scene reconstruction and visualization from community photo collections,” Proceedings of the IEEE, vol. 98, no. 8, Article ID 5483186, pp. 1370–1390, 2010. View at Publisher · View at Google Scholar · View at Scopus
  54. H. C. Longuet-higgins, “A computer algorithm for reconstructing a scene from two projections,” Nature, vol. 293, no. 5828, pp. 133–135, 1981. View at Publisher · View at Google Scholar · View at Scopus
  55. C. Tomasi and T. Kanade, “Shape and motion from image streams under orthography: a factorization method,” International Journal of Computer Vision, vol. 9, no. 2, pp. 137–154, 1992. View at Publisher · View at Google Scholar · View at Scopus
  56. C. J. Poelman and T. Kanade, “A paraperspective factorization method for shape and motion recovery,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 3, pp. 206–218, 1997. View at Scopus
  57. P. Sturm and B. Triggs, “A factorization based algorithm for multi-image projective structure and motion,” in Proceedings of the 4th European Conference on Computer Vision, 1996.
  58. J. P. Tardif, A. Bartoli, M. Trudeau, N. Guilbert, and S. Roy, “Algorithms for batch matrix factorization with application to structure-from-motion,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '07), June 2007. View at Publisher · View at Google Scholar · View at Scopus
  59. B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon, “Bundle adjustment—a modern synthesis,” in Proceedings of the International Workshop on Vision Algorithms, pp. 298–372, 1999.
  60. E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd, “Generic and real-time structure from motion using local bundle adjustment,” Image and Vision Computing, vol. 27, no. 8, pp. 1178–1193, 2009. View at Publisher · View at Google Scholar · View at Scopus
  61. G. Zhang, J. Jia, T. T. Wong, and H. Bao, “Consistent depth maps recovery from a video sequence,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 6, pp. 974–988, 2009. View at Publisher · View at Google Scholar · View at Scopus
  62. T. Jebara, A. Azarbayejani, and A. Pentland, “3D structure from 2D motion,” IEEE Signal Processing Magazine, vol. 16, no. 3, pp. 66–84, 1999. View at Publisher · View at Google Scholar · View at Scopus
  63. G. Wang, H. T. Tsui, and Q. M. Jonathan Wu, “What can we learn about the scene structure from three orthogonal vanishing points in images,” Pattern Recognition Letters, vol. 30, no. 3, pp. 192–202, 2009. View at Publisher · View at Google Scholar · View at Scopus
  64. S. N. Sinha, D. Steedly, and R. Szeliski, “A multi-stage linear approach to structure from motion,” in Proceedings of the European Conference on Computer Vision (ECCV '10), 2010.
  65. J. I. Thomas and J. Oliensis, “Dealing with noise in multiframe structure from motion,” Computer Vision and Image Understanding, vol. 76, no. 2, pp. 109–124, 1999. View at Publisher · View at Google Scholar · View at Scopus
  66. S. Carlsson and D. Weinshall, “Dual computation of projective shape and camera positions from multiple images,” International Journal of Computer Vision, vol. 27, no. 3, pp. 227–241, 1998. View at Scopus
  67. D. Crandall, A. Owens, N. Snavely, and D. Huttenlocher, “Discrete-continuous optimization for large-scale structure from motion,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '11), 2011.
  68. M. Havlena, A. Torii, and T. Pajdla, “Efficient structure from motion by graph optimization,” in Proceedings of the European Conference on Computer Vision (ECCV '10), 2010.
  69. R. Gherardi, M. Farenzena, and A. Fusello, “Improving the efficiency of hierarchical structure-and-moton,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '10), pp. 1594–1600, June 2010. View at Publisher · View at Google Scholar · View at Scopus
  70. R. Roberts, S. Sinha, R. Szeliski, and D. Steedly, “Structure from motion for scenes with large duplicate structures,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '11), 2011.
  71. T. Fang and L. Quan, “Resampling structure from motion,” in Proceedings of the European Conference on Computer Vision (ECCV '10), 2010.
  72. S. Savarese and S. Y. Z. Bao, “Semantic structure from motion,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '11), 2011.
  73. A. Bartoli and P. Sturm, “Structure-from-motion using lines: representation, triangulation, and bundle adjustment,” Computer Vision and Image Understanding, vol. 100, no. 3, pp. 416–441, 2005. View at Publisher · View at Google Scholar · View at Scopus
  74. M. E. Spetsakis and J. Aloimonos, “Structure from motion using line correspondences,” International Journal of Computer Vision, vol. 4, no. 3, pp. 171–183, 1990. View at Publisher · View at Google Scholar · View at Scopus
  75. C. J. Taylor and D. J. Kriegman, “Structure and motion from line segments in multiple images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 11, pp. 1021–1032, 1995. View at Publisher · View at Google Scholar · View at Scopus
  76. R. I. Hartley, “Linear method for reconstruction from lines and points,” in Proceedings of the 5th International Conference on Computer Vision (ICCV '95), pp. 882–887, June 1995. View at Scopus
  77. D. Tubic, P. Hebert, and D. Laurendeau, “3D surface modeling from curves,” Image and Vision Computing, vol. 22, no. 9, pp. 719–734, 2004. View at Publisher · View at Google Scholar · View at Scopus
  78. J. E. Solem and A. Heyden, “Reconstructing open surfaces from image data,” International Journal of Computer Vision, vol. 69, no. 3, pp. 267–275, 2006. View at Publisher · View at Google Scholar · View at Scopus
  79. J. Y. Kaminski and A. Shashua, “Multiple view geometry of general algebraic curves,” International Journal of Computer Vision, vol. 56, no. 3, pp. 195–219, 2004. View at Publisher · View at Google Scholar · View at Scopus
  80. R. Berthilsson, K. Astrom, and A. Heyden, “Reconstruction of general curves, using factorization and bundle adjustment,” International Journal of Computer Vision, vol. 41, no. 3, pp. 171–182, 2001. View at Publisher · View at Google Scholar · View at Scopus
  81. C. Liang and K. Y. K. Wong, “3D reconstruction using silhouettes from unordered viewpoints,” Image and Vision Computing, vol. 28, no. 4, pp. 579–589, 2010. View at Publisher · View at Google Scholar · View at Scopus
  82. R. Hartley and F. Kahl, “Critical configurations for projective reconstruction from multiple views,” International Journal of Computer Vision, vol. 71, no. 1, pp. 5–47, 2007. View at Publisher · View at Google Scholar · View at Scopus
  83. T. Joshi, N. Ahuja, and J. Ponce, “Structure and motion estimation from dynamic silhouettes under perspective projection,” International Journal of Computer Vision, vol. 31, no. 1, pp. 31–50, 1999. View at Publisher · View at Google Scholar · View at Scopus
  84. X. Liu, H. Yao, and W. Gao, “Shape from silhouette outlines using an adaptive dandelion model,” Computer Vision and Image Understanding, vol. 105, no. 2, pp. 121–130, 2007. View at Publisher · View at Google Scholar · View at Scopus
  85. Y. Yemez and C. J. Wetherilt, “A volumetric fusion technique for surface reconstruction from silhouettes and range data,” Computer Vision and Image Understanding, vol. 105, no. 1, pp. 30–41, 2007. View at Publisher · View at Google Scholar · View at Scopus
  86. N. Snavely, S. M. Seitz, and R. Szeliski, “Modeling the world from Internet photo collections,” International Journal of Computer Vision, vol. 80, no. 2, pp. 189–210, 2008. View at Publisher · View at Google Scholar · View at Scopus
  87. S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A comparison and evaluation of multi-view stereo reconstruction algorithms,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '06), pp. 519–526, June 2006. View at Publisher · View at Google Scholar · View at Scopus
  88. I. Simon, N. Snavely, and S. M. Seitz, “Scene summarization for online image collections,” in Proceedings of the IEEE 11th International Conference on Computer Vision (ICCV '07), October 2007. View at Publisher · View at Google Scholar · View at Scopus
  89. S. Agarwal, Y. Furukawa, N. Snavely, B. Curless, S. M. Seitz, and R. Szeliski, “Reconstructing Rome,” Computer, vol. 43, no. 6, pp. 40–47, 2010. View at Publisher · View at Google Scholar · View at Scopus
  90. G. Zhang, Z. Dong, J. Jia, T. T. Wong, and H. Bao, “Efficient non-consecutive feature tracking for structure-from-motion,” in Proceedings of the European Conference on Computer Vision (ECCV '10), 2010.
  91. E. Marchand and F. Chaumette, “Active vision for complete scene reconstruction and exploration,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 1, pp. 65–72, 1999. View at Scopus
  92. F. Dornaika and C. K. R. Chung, “Stereo geometry from 3-D ego-motion streams,” IEEE Transactions on Systems, Man, and Cybernetics, Part B, vol. 33, no. 2, pp. 308–323, 2003. View at Publisher · View at Google Scholar · View at Scopus
  93. O. Koch and S. Teller, “Wide-area egomotion estimation from known 3D structure,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '07), pp. 437–444, June 2007. View at Publisher · View at Google Scholar · View at Scopus
  94. P. Debevec, C. Tchou, et al., “Estimating surface reflectance properties of a complex scene under captured natural illumination,” Tech. Rep. ICT-TR-06.2004, University of Southern California Institute for Creative Technologies, Marina del Rey, Calif, USA, 2004.
  95. N. Snavely, S. M. Seitz, and R. Szeliski, “Skeletal graphs for efficient structure from motion,” in Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition (CVPR '08), June 2008. View at Publisher · View at Google Scholar · View at Scopus