Table of Contents Author Guidelines Submit a Manuscript
Journal of Sensors
Volume 2018, Article ID 2075057, 12 pages
https://doi.org/10.1155/2018/2075057
Research Article

Spark Sensing: A Cloud Computing Framework to Unfold Processing Efficiencies for Large and Multiscale Remotely Sensed Data, with Examples on Landsat 8 and MODIS Data

1Department of Computer Science and Engineering, Tandon School of Engineering, New York University, Brooklyn, NY 11201, USA
2Center for Urban Science + Progress, New York University, Brooklyn, NY 11201, USA

Correspondence should be addressed to Hai Lan; ude.uyn@nal.iah

Received 26 April 2018; Accepted 8 July 2018; Published 23 August 2018

Academic Editor: Victor Mesev

Copyright © 2018 Hai Lan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. M. F. Goodchild, H. Guo, A. Annoni et al., “Next-generation digital earth,” Proceedings of the National Academy of Sciences, vol. 109, no. 28, pp. 11088–11094, 2012. View at Publisher · View at Google Scholar · View at Scopus
  2. C. Yang, M. Yu, F. Hu, Y. Jiang, and Y. Li, “Utilizing cloud computing to address big geospatial data challenges,” Computers, Environment and Urban Systems, vol. 61, pp. 120–128, 2017. View at Publisher · View at Google Scholar · View at Scopus
  3. C. A. Lee, S. D. Gasster, A. Plaza, C. I. Chang, and B. Huang, “Recent developments in high performance computing for remote sensing: a review,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 4, no. 3, pp. 508–527, 2011. View at Publisher · View at Google Scholar · View at Scopus
  4. M. A. Wulder and N. C. Coops, “Make earth observations open access: freely available satellite imagery will improve science and environmental-monitoring products,” Nature, vol. 513, no. 7516, p. 30, 2014. View at Google Scholar
  5. Z. Wang, J. Zhong, H. Lan, Z. Wang, and Z. Sha, “Association analysis between spatiotemporal variation of net primary productivity and its driving factors in Inner Mongolia, China during 1994–2013,” Ecological Indicators, 2017. View at Publisher · View at Google Scholar · View at Scopus
  6. Y. Ma, H. Wu, L. Wang et al., “Remote sensing big data computing: challenges and opportunities,” Future Generation Computer Systems, vol. 51, pp. 47–60, 2015. View at Publisher · View at Google Scholar · View at Scopus
  7. Satellite Imaging Corporation, “WorldView-4 satellite image gallery,” March 2018, https://www.satimagingcorp.com/gallery/worldview-4/.
  8. Satellite Imaging Corporation, “Quick Bird satellite sensor,” March 2017, https://www.satimagingcorp.com/satellite-sensors/quickbird/.
  9. H. Lan and Y. Xie, “A semi-ellipsoid-model based fuzzy classifier to map grassland in Inner Mongolia, China,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 85, pp. 21–31, 2013. View at Publisher · View at Google Scholar · View at Scopus
  10. M. A. Friedl, D. K. McIver, J. C. F. Hodges et al., “Global land cover mapping from MODIS: algorithms and early results,” Remote Sensing of Environment, vol. 83, no. 1-2, pp. 287–302, 2002. View at Publisher · View at Google Scholar · View at Scopus
  11. US Geological Survey, “Sensors-Hyperion,” March 2018, https://eo1.usgs.gov/sensors/hyperion.
  12. W. Turner, S. Spector, N. Gardiner, M. Fladeland, E. Sterling, and M. Steininger, “Remote sensing for biodiversity science and conservation,” Trends in Ecology & Evolution, vol. 18, no. 6, pp. 306–314, 2003. View at Publisher · View at Google Scholar · View at Scopus
  13. Q. Huang, C. Yang, K. Benedict, S. Chen, A. Rezgui, and J. Xie, “Utilize cloud computing to support dust storm forecasting,” International Journal of Digital Earth, vol. 6, no. 4, pp. 338–355, 2013. View at Publisher · View at Google Scholar · View at Scopus
  14. E. Amazon, “Amazon web services,” 2015, November 2012, http://aws.amazon.com/es/ec2/. View at Google Scholar
  15. B. Wilder, Cloud Architecture Patterns: Using Microsoft Azure, O'Reilly Media, Inc., 2012.
  16. D. Sanderson, Programming Google App Engine: Build and Run Scalable Web Apps on Google’s Infrastructure, O'Reilly Media, Inc., 2009.
  17. J. Dean and S. Ghemawat, “MapReduce: simplified data processing on large clusters,” Communications of the ACM, vol. 51, no. 1, pp. 107–113, 2008. View at Publisher · View at Google Scholar · View at Scopus
  18. Apache Software Foundation, “Apache Spark,” April 2018, https://spark.apache.org/.
  19. M. Guller, Big Data Analytics with Spark: a practitioner’s Guide to Using Spark for Large Scale Data Analysis, Springer, 2015.
  20. G. Cavallaro, M. Riedel, C. Bodenstein et al., “Scalable developments for big data analytics in remote sensing,” in 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 1366–1369, Milan, Italy, July 2015. View at Publisher · View at Google Scholar · View at Scopus
  21. L. Wang, Y. Ma, J. Yan, V. Chang, and A. Y. Zomaya, “pipsCloud: high performance cloud computing for remote sensing big data management and processing,” Future Generation Computer Systems, vol. 78, pp. 353–368, 2018. View at Publisher · View at Google Scholar · View at Scopus
  22. X. Chen and L. Zhou, “The remote sensing image segmentation mean shift algorithm parallel processing based on MapReduce,” in International Conference on Intelligent Earth Observing and Applications 2015, 98083T, Guilin, China, December 2015. View at Publisher · View at Google Scholar · View at Scopus
  23. R. Giachetta, “A framework for processing large scale geospatial and remote sensing data in MapReduce environment,” Computers & Graphics, vol. 49, pp. 37–46, 2015. View at Publisher · View at Google Scholar · View at Scopus
  24. Z. Sun, F. Chen, M. Chi, and Y. Zhu, “A spark-based big data platform for massive remote sensing data processing,” in ICDS 2015 Proceedings of the Second International Conference on Data Science, pp. 120–126, Sydney, Australia, August 2015. View at Publisher · View at Google Scholar · View at Scopus
  25. W. Huang, L. Meng, D. Zhang, and W. Zhang, “In-memory parallel processing of massive remotely sensed data using an apache spark on Hadoop YARN model,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 10, no. 1, pp. 3–19, 2017. View at Publisher · View at Google Scholar · View at Scopus
  26. X. Fan, B. Lang, Y. Zhou, and T. Zang, “Adding network bandwidth resource management to Hadoop YARN,” in 2017 Seventh International Conference on Information Science and Technology (ICIST), pp. 444–449, Da Nang, Vietnam, April 2017. View at Publisher · View at Google Scholar · View at Scopus
  27. C.-Z. Qin, L.-J. Zhan, and A.-X. Zhu, “How to apply the geospatial data abstraction library (GDAL) properly to parallel geospatial raster I/O?” Transactions in GIS, vol. 18, no. 6, pp. 950–957, 2014. View at Publisher · View at Google Scholar · View at Scopus
  28. J. W. Rouse Jr., R. H. Haas, J. A. Schell, and D. W. Deering, “Monitoring vegetation systems in the Great Plains with ERTS,” in NASA. Goddard Space Flight Center 3d ERTS-1 Symp, College Station, TX, USA, January 1974.
  29. S. K. McFeeters, “The use of the normalized difference water index (NDWI) in the delineation of open water features,” International Journal of Remote Sensing, vol. 17, no. 7, pp. 1425–1432, 1996. View at Publisher · View at Google Scholar · View at Scopus
  30. A. C. Frery and T. Perciano, “Image data formats and color representation,” in Introduction to Image Processing Using R. SpringerBriefs in Computer Science, pp. 21–29, Springer, London, 2013. View at Publisher · View at Google Scholar · View at Scopus
  31. US Department of the Interior, and US Geological Survey, “Landsat 8,” April 2018, https://landsat.usgs.gov/landsat-8.
  32. Amazon Web Services Inc, “Landsat on AWS,” March 2018, https://aws.amazon.com/public-datasets/landsat/.
  33. Amazon Web Services Inc, “MODIS on AWS,” June 2018, https://docs.opendata.aws/modis-pds/readme.html.
  34. Suez Canal Authority, “New Suez Canal,” June 2017; https://www.suezcanal.gov.eg/English/About/SuezCanal/Pages/NewSuezCanal.aspx.
  35. N. Gorelick, M. Hancher, M. Dixon, S. Ilyushchenko, D. Thau, and R. Moore, “Google Earth Engine: planetary-scale geospatial analysis for everyone,” Remote Sensing of Environment, vol. 202, pp. 18–27, 2017. View at Publisher · View at Google Scholar · View at Scopus
  36. P. S. Chavez, “Image-based atmospheric corrections-revisited and improved,” Photogrammetric Engineering and Remote Sensing, vol. 62, no. 9, pp. 1025–1035, 1996. View at Google Scholar
  37. Apache Software Foundation, “Tuning Spark,” June 2018, http://spark.apache.org/docs/latest/tuning.html#tuning-spark.