Table of Contents Author Guidelines Submit a Manuscript
Scientific Programming
Volume 2015 (2015), Article ID 680271, 13 pages
http://dx.doi.org/10.1155/2015/680271
Research Article

Scheduling Multilevel Deadline-Constrained Scientific Workflows on Clouds Based on Cost Optimization

1Department of Computer Science, AGH University of Science and Technology, Aleja Mickiewicza 30, 30-059 Kraków, Poland
2ACC CYFRONET AGH, Ulica Nawojki 11, 30-950 Kraków, Poland
3USC Information Sciences Institute, 4676 Admiralty Way, Marina del Rey, CA 90292, USA
4Center for Research Computing, University of Notre Dame, Notre Dame, IN 46556, USA

Received 15 May 2014; Accepted 22 November 2014

Academic Editor: Roman Wyrzykowski

Copyright © 2015 Maciej Malawski et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. E. Deelman, G. Juve, M. Malawski, and J. Nabrzyski, “Hosted science: managing computational workflows in the cloud,” Parallel Processing Letters, vol. 23, no. 2, Article ID 1340004, 2013. View at Publisher · View at Google Scholar · View at Scopus
  2. AWS, “AWS public datasets,” 2013, http://aws.amazon.com/publicdatasets/.
  3. M. Malawski, G. Juve, E. Deelman, and J. Nabrzyski, “Cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds,” in Proceedings of the 24th International Conference for High Performance Computing, Networking, Storage and Analysis (SC '12), IEEE, November 2012. View at Publisher · View at Google Scholar · View at Scopus
  4. M. Bubak, M. Kasztelnik, M. Malawski, J. Meizner, P. Nowakowski, and S. Varma, “Evaluation of cloud providers for VPH applications,” in Proceedings of the 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CC-Grid '13), May 2013.
  5. R. Fourer, D. M. Gay, and B. W. Kernighan, AMPL: A Modeling Language for Mathematical Programming, Duxbury Press, 2002.
  6. M. Steglich, “CMPL (Coin mathematical programming language),” 2014, https://projects.coin-or.org/Cmpl.
  7. M. Malawski, K. Figiela, and J. Nabrzyski, “Cost minimization for computational applications on hybrid cloud infrastructures,” Future Generation Computer Systems, vol. 29, no. 7, pp. 1786–1794, 2013. View at Publisher · View at Google Scholar · View at Scopus
  8. S. Bharathi, A. Chervenak, E. Deelman, G. Mehta, M.-H. Su, and K. Vahi, “Characterization of scientific workflows,” in Proceedings of the 3rd Workshop on Workflows in Support of Large-Scale Science (WORKS '08), pp. 1–10, IEEE, November 2008. View at Publisher · View at Google Scholar · View at Scopus
  9. R. Duan, R. Prodan, and X. Li, “A sequential cooperative game theoretic approach to storage-aware scheduling of multiple large-scale workflow applications in grids,” in Proceedings of the 13th ACM/IEEE International Conference on Grid Computing (Grid '12), pp. 31–39, IEEE, September 2012. View at Publisher · View at Google Scholar · View at Scopus
  10. M. Mao and M. Humphrey, “Auto-scaling to minimize cost and meet application deadlines in cloud workflows,” in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC '11), ACM, New York, NY, USA, November 2011. View at Publisher · View at Google Scholar · View at Scopus
  11. M. Malawski, K. Figiela, M. Bubak, E. Deelman, and J. Nabrzyski, “Cost optimization of execution of multi-level deadline-constrained scientific workows on clouds,” in Parallel Processing and Applied Mathematics—10th International Conference, PPAM 2013, Warsaw, Poland, September 8–11, 2013, Revised Selected Papers, Part I, vol. 8384 of Lecture Notes in Computer Science, pp. 251–260, Springer, Berlin, Germany, 2014. View at Google Scholar
  12. S. Abrishami, M. Naghibzadeh, and D. H. J. Epema, “Deadline-constrained workow scheduling algorithms for infrastructure as a service clouds,” Future Generation Computer Systems, vol. 29, no. 1, pp. 158–169, 2013. View at Publisher · View at Google Scholar · View at Scopus
  13. J. J. Durillo, H. M. Fard, and R. Prodan, “MOHEFT: a multi-objective list-based method for workflow scheduling,” in Proceedings of the 4th IEEE International Conference on Cloud Computing Technology and Science (CloudCom '12), pp. 185–192, Taipei, Taiwan, December 2012. View at Publisher · View at Google Scholar · View at Scopus
  14. L. F. Bittencourt and E. R. M. Madeira, “HCOC: a cost optimization algorithm for workflow scheduling in hybrid clouds,” Journal of Internet Services and Applications, vol. 2, no. 3, pp. 207–227, 2011. View at Publisher · View at Google Scholar · View at Scopus
  15. R. van den Bossche, K. Vanmechelen, and J. Broeckhove, “Online cost-efficient scheduling of deadline-constrained workloads on hybrid clouds,” Future Generation Computer Systems, vol. 29, no. 4, pp. 973–985, 2013. View at Publisher · View at Google Scholar · View at Scopus
  16. S. Pandey, A. Barker, K. K. Gupta, and R. Buyya, “Minimizing execution costs when using globally distributed Cloud services,” in Proceedings of the 24th IEEE International Conference on Advanced Information Networking and Applications, pp. 222–229, April 2010. View at Publisher · View at Google Scholar · View at Scopus
  17. J. Chen, C. Wang, B. B. Zhou, L. Sun, Y. C. Lee, and A. Y. Zomaya, “Tradeoffs between profit and customer satisfaction for service provisioning in the cloud,” in Proceedings of the 20th International Symposium on High Performance Distributed Computing (HPDC '11), pp. 229–238, ACM, San Jose, Calif, USA, 2011.
  18. H. Kim, Y. El-Khamra, I. Rodero, S. Jha, and M. Parashar, “Autonomic management of application workflows on hybrid computing infrastructure,” Scientific Programming, vol. 19, no. 2-3, pp. 75–89, 2011. View at Publisher · View at Google Scholar · View at Scopus
  19. R. Tolosana-Calasanz, J. Á. Bañares, C. Pham, and O. F. Rana, “Enforcing QoS in scientific workflow systems enacted over Cloud infrastructures,” Journal of Computer and System Sciences, vol. 78, no. 5, pp. 1300–1315, 2012. View at Publisher · View at Google Scholar · View at Scopus
  20. T. A. L. Genez, L. F. Bittencourt, and E. R. M. Madeira, “Using time discretization to schedule scientific workflows in multiple cloud providers,” in Proceedings of the IEEE 6th International Conference on Cloud Computing (CLOUD '13), pp. 123–130, Santa Clara, Calif, USA, July 2013. View at Publisher · View at Google Scholar · View at Scopus
  21. IBM, “IBM ILOG CPLEX Optimization Studio CPLEX User's Manual,” 2013, http://pic.dhe.ibm.com/infocenter/cosinfoc/v12r5/topic/ilog.odms.studio.help/pdf/usrcplex.pdf.
  22. J. Forrest, “Cbc (coin-or branch and cut) open-source mixed integer programming solver,” 2012, https://projects.coin-or.org/Cbc.
  23. G. Juve, A. Chervenak, E. Deelman, S. Bharathi, G. Mehta, and K. Vahi, “Characterizing and profiling scientific workflows,” Future Generation Computer Systems, vol. 29, no. 3, pp. 682–692, 2013. View at Publisher · View at Google Scholar · View at Scopus
  24. CloudHarmony, “Benchmarks,” 2014, http://cloudharmony.com/benchmarks.
  25. USC epigenome center, http://epigenome.usc.edu.
  26. J. Livny, H. Teonadi, M. Livny, and M. K. Waldor, “High-throughput, kingdom-wide prediction and annotation of bacterial non-coding RNAs,” PLoS ONE, vol. 3, no. 9, Article ID e3197, 2008. View at Publisher · View at Google Scholar · View at Scopus
  27. G. B. Berriman, E. Deelman, J. C. Good et al., “Montage: a grid enabled engine for delivering custom science-grade mosaics on demand,” in Optimizing Scientific Return for Astronomy through Information Technologies, vol. 5493 of Proceedings of SPIE, pp. 221–232, June 2004. View at Publisher · View at Google Scholar · View at Scopus
  28. P. Maechling, E. Deelman, L. Zhao et al., “SCEC cyber-shake workows|automating probabilistic seismic hazard analysis calculations,” in Workows for e-Science, I. Taylor, E. Deelman, D. Gannon, and M. Shields, Eds., pp. 143–163, Springer, London, UK, 2007. View at Google Scholar
  29. A. Abramovici, W. E. Althouse, R. W. P. Drever et al., “LIGO: the laser interferometer gravitational-wave observatory,” Science, vol. 256, no. 5055, pp. 325–333, 1992. View at Publisher · View at Google Scholar · View at Scopus
  30. Workflow Generator, 2014, https://confluence.pegasus.isi.edu/display/pegasus/WorkflowGenerator.
  31. B. Balis, “Hypermedia workflow: a new approach to Data-Driven scientific workflows,” in Proceedings of the SC Companion: High Performance Computing, Networking Storage and Analysis (SCC '12), pp. 100–107, November 2012. View at Publisher · View at Google Scholar · View at Scopus
  32. Cloud Harmony, “What is ECU? CPU benchmarking in Cloud,” 2010, http://blog.cloudharmony.com/2010/05/what-is-ecu-cpu-benchmarking-in-cloud.html.
  33. R. F. da Silva, G. Juve, E. Deelman et al., “Toward fine-grained online task characteristics estimation in scientific workflows,” in Proceedings of the 8th Workshop on Workows in Support of Large-Scale Science (WORKS '13), pp. 58–67, ACM, Denver, Colo, USA, November 2013. View at Publisher · View at Google Scholar
  34. R. Van Den Bossche, K. Vanmechelen, and J. Broeckhove, “Cost-optimal scheduling in hybrid IaaS clouds for deadline constrained workloads,” in Proceedings of the 3rd IEEE International Conference on Cloud Computing (CLOUD '10), pp. 228–235, Miami, Fla, USA, July 2010. View at Publisher · View at Google Scholar · View at Scopus