Table of Contents Author Guidelines Submit a Manuscript
Scientific Programming
Volume 2016, Article ID 7241928, 13 pages
http://dx.doi.org/10.1155/2016/7241928
Research Article

Feedback-Based Resource Allocation in MapReduce-Based Systems

1OEG, ETS de Ingenieros Informáticos, Universidad Politécnica de Madrid, Campus de Montegancedo, s/n Boadilla del Monte, 28660 Madrid, Spain
2Inria Rennes-Bretagne Atlantique Research Centre, Campus Universitaire de Beaulieu, Rennes, 35042 Brittany, France

Received 14 January 2016; Accepted 28 March 2016

Academic Editor: Zhihui Du

Copyright © 2016 Bunjamin Memishi et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. J. Dean, S. Ghemawat, and Google Inc, “MapReduce: simplified data processing on large clusters,” in Proceedings of the 6th Conference on Symposium on Operating Systems Design & Implementation (OSDI '04), USENIX Association, San Francisco, Calif, USA, December 2004.
  2. V. K. Vavilapalli, A. C. Murthy, C. Douglas et al., “Apache hadoop YARN: yet another resource negotiator,” in Proceedings of the 4th Annual Symposium on Cloud Computing (SoCC '13), pp. 5:1–5:16, ACM, Santa Clara, Calif, USA, 2013.
  3. N. R. Herbst, S. Kounev, and R. Reussner, “Elasticity in cloud computing: what it is, and what it is not,” in Proceedings of the 10th International Conference on Autonomic Computing (ICAC '13), pp. 23–27, USENIX, San Jose, Calif, USA, 2013.
  4. K. J. Astrom and R. M. Murray, Feedback Systems: An Introduction for Scientists and Engineers, Princeton University Press, Princeton, NJ, USA, 2008.
  5. M. Armbrust, A. Fox, R. Griffith et al., “Above the clouds: a berkeley view of cloud computing,” Tech. Rep., University of California, Berkeley, Calif, USA, 2009. View at Google Scholar
  6. J. Montes, A. Sánchez, B. Memishi, M. S. Pérez, and G. Antoniu, “GMonE: a complete approach to cloud monitoring,” Future Generation Computer Systems, vol. 29, no. 8, pp. 2026–2040, 2013. View at Publisher · View at Google Scholar · View at Scopus
  7. A. Verma, B. Cho, N. Zea, I. Gupta, and R. H. Campbell, “Breaking the MapReduce stage barrier,” Cluster Computing, vol. 16, no. 1, pp. 191–206, 2013. View at Publisher · View at Google Scholar · View at Scopus
  8. T.-C. Huang, K.-C. Chu, W.-T. Lee, and Y.-S. Ho, “Adaptive combiner for MapReduce on cloud computing,” Cluster Computing, vol. 17, no. 4, pp. 1231–1252, 2014. View at Google Scholar
  9. F. Salfner, M. Lenk, and M. Malek, “A survey of online failure prediction methods,” ACM Computing Surveys, vol. 42, no. 3, article 10, 2010. View at Publisher · View at Google Scholar · View at Scopus
  10. S. Y. Ko, I. Hoque, B. Cho, and I. Gupta, “Making cloud intermediate data fault-tolerant,” in Proceedings of the 1st ACM Symposium on Cloud Computing (SoCC '10), pp. 181–192, ACM, New York, NY, USA, June 2010. View at Publisher · View at Google Scholar · View at Scopus
  11. F. Dinu and T. S. Eugene Ng, “Understanding the effects and implications of compute node related failures in Hadoop,” in Proceedings of the 21st ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC '12), pp. 187–197, Delft, The Netherlands, June 2012. View at Publisher · View at Google Scholar · View at Scopus
  12. R. Appuswamy, C. Gkantsidis, D. Narayanan, O. Hodson, and A. Rowstron, “Scale-up vs scale-out for hadoop: time to rethink?” in Proceedings of the 4th Annual Symposium on Cloud Computing (SOCC '13), pp. 20.1–20.13, ACM, New York, NY, USA, 2013.
  13. G. Ananthanarayanan, A. Ghodsi, A. Wang et al., “PACMan: coordinated memory caching for parallel jobs,” in Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation (NSDI '12), p. 20, USENIX Association, Berkeley, Calif, USA, 2012.
  14. K. Elmeleegy, “Piranha: optimizing short jobs in hadoop,” Proceedings of the VLDB Endowment, vol. 6, no. 11, pp. 985–996, August 2013. View at Publisher · View at Google Scholar
  15. T. White, Hadoop: The Definitive Guide: Storage and Analysis at Internet Scale, O'Reilly, 3rd edition, 2012.
  16. Apache Software Foundation, Apache Hadoop NextGen MapReduce (YARN), 2015, http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html.
  17. Hortonworks, Hortonworks Data Platform: Installing HDP Manually, 2013.
  18. M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, and I. Stoica, “Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling,” in Proceedings of the 5th ACM EuroSys Conference on Computer Systems (EuroSys '10), pp. 265–278, ACM, Paris, France, April 2010. View at Publisher · View at Google Scholar · View at Scopus
  19. S. Babu, “Towards automatic optimization of MapReduce programs,” in Proceedings of the Proceedings of the 1st ACM Symposium on Cloud Computing, pp. 137–142, ACM, New York, NY, USA, June 2010. View at Publisher · View at Google Scholar · View at Scopus
  20. E. Jahani, M. J. Cafarella, and C. Ré, “Automatic optimization for MapReduce programs,” Proceedings of the VLDB Endowment, vol. 4, no. 6, pp. 385–396, 2011. View at Publisher · View at Google Scholar
  21. Z. Tang, J. Zhou, K. Li, and R. Li, “A MapReduce task scheduling algorithm for deadline constraints,” Cluster Computing, vol. 16, no. 4, pp. 651–662, 2013. View at Publisher · View at Google Scholar · View at Scopus
  22. Y. Guo, J. Rao, C. Jiang, and X. Zhou, “FlexSlot: moving hadoop into the cloud with flexible slot management,” in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC '14), pp. 959–969, IEEE Press, New Orleans, La, USA, November 2014. View at Publisher · View at Google Scholar · View at Scopus
  23. S. Tang, B.-S. Lee, and B. He, “DynamicMR: a dynamic slot allocation optimization framework for mapreduce clusters,” IEEE Transactions on Cloud Computing, vol. 2, no. 3, pp. 333–347, 2014. View at Publisher · View at Google Scholar · View at Scopus
  24. M. Li, L. Zeng, S. Meng et al., “MRONLINE: mapReduce online performance tuning,” in Proceedings of the ACM 23rd International Symposium on High-Performance Parallel and Distributed Computing (HPDC '14), pp. 165–176, New York, NY, USA, 2014. View at Publisher · View at Google Scholar
  25. B. Palanisamy, A. Singh, L. Liu, and B. Langston, “Cura: a cost-optimized model for MapReduce in a cloud,” in Proceedings of the 27th IEEE International Parallel and Distributed Processing Symposium (IPDPS '13), pp. 1275–1286, IEEE, Boston, Mass, USA, May 2013. View at Publisher · View at Google Scholar · View at Scopus
  26. A. Clement, M. Kapritsos, M. Kapritsos et al., “Upright cluster services,” in Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles (SOSP '09), pp. 277–290, New York, NY, USA, 2009. View at Publisher · View at Google Scholar
  27. P. Costa, M. Pasin, A. Bessani, and M. Correia, “Byzantine fault-tolerant MapReduce: faults are not just crashes,” in Proceedings of the 3rd IEEE International Conference on Cloud Computing Technology and Science (CLOUDCOM '11), pp. 17–24, IEEE Computer Society, Washington, DC, USA, 2011.
  28. M. Correia, P. Costa, M. Pasin, A. Bessani, F. Ramos, and P. Verissimo, “On the feasibility of byzantine fault-tolerant mapreduce in clouds-of-clouds,” in Proceedings of the 31st Symposium on Reliable Distributed Systems (SRDS '12), pp. 448–453, IEEE, Irvine, Calif, USA, October 2012. View at Publisher · View at Google Scholar · View at Scopus
  29. Z. Xiao and Y. Xiao, “Achieving accountable MapReduce in cloud computing,” Future Generation Computer Systems, vol. 30, no. 1, pp. 1–13, 2014. View at Publisher · View at Google Scholar · View at Scopus
  30. F. Wang, J. Qiu, J. Yang, B. Dong, X. Li, and Y. Li, “Hadoop high availability through metadata replication,” in Proceedings of the 1st International Workshop on Cloud Data Management (CloudDB '09), pp. 37–44, ACM, Hong Kong, November 2009. View at Publisher · View at Google Scholar · View at Scopus
  31. H. Lin, X. Ma, and W.-C. Feng, “Reliable MapReduce computing on opportunistic resources,” Cluster Computing, vol. 15, no. 2, pp. 145–161, 2012. View at Publisher · View at Google Scholar · View at Scopus
  32. N. Chohan, C. Castillo, M. Spreitzer, M. Steinder, A. Tantawi, and C. Krintz, “See Spot Run: using spot instances for mapreduce workflows,” in Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing (HotCloud '10), p. 7, USENIX Association, Berkeley, Calif, USA, 2010.
  33. H. Liu, “Cutting mapReduce cost with spot market,” in Proceedings of the 3rd USENIX Conference on Hot Topics in Cloud Computing (HotCloud '11), p. 5, Berkeley, Calif, USA, 2011.