Table of Contents
Advances in Software Engineering
Volume 2010 (2010), Article ID 620836, 18 pages
http://dx.doi.org/10.1155/2010/620836
Research Article

Software Test Automation in Practice: Empirical Observations

Department of Information Technology, Laboratory of Software Engineering, Lappeenranta University of Technology, P.O. Box 20, 53851 Lappeenranta, Finland

Received 10 June 2009; Revised 28 August 2009; Accepted 5 November 2009

Academic Editor: Phillip Laplante

Copyright © 2010 Jussi Kasurinen et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. E. Kit, Software Testing in the Real World: Improving the Process, Addison-Wesley, Reading, Mass, USA, 1995.
  2. G. Tassey, “The economic impacts of inadequate infrastructure for software testing,” RTI Project 7007.011, U.S. National Institute of Standards and Technology, Gaithersburg, Md, USA, 2002. View at Google Scholar
  3. R. Ramler and K. Wolfmaier, “Observations and lessons learned from automated testing,” in Proceedings of the International Workshop on Automation of Software Testing (AST '06), pp. 85–91, Shanghai, China, May 2006.
  4. K. Karhu, T. Repo, O. Taipale, and K. Smolander, “Empirical observations on software testing automation,” in Proceedings of the 2nd International Conference on Software Testing, Verification, and Validation (ICST '09), pp. 201–209, Denver, Colo, USA, April 2009. View at Publisher · View at Google Scholar
  5. O. Taipale and K. Smolander, “Improving software testing by observing causes, effects, and associations from practice,” in Proceedings of the International Symposium on Empirical Software Engineering (ISESE '06), Rio de Janeiro, Brazil, September 2006.
  6. B. Shea, “Sofware testing gets new respect,” InformationWeek, July 2000. View at Google Scholar
  7. E. Dustin, J. Rashka, and J. Paul, Automated Software Testing: Introduction, Management, and Performance, Addison-Wesley, Boston, Mass, USA, 1999.
  8. S. Berner, R. Weber, and R. K. Keller, “Observations and lessons learned from automated testing,” in Proceedings of the 27th International Conference on Software Engineering (ICSE '05), pp. 571–579, St. Louis, Mo, USA, May 2005.
  9. J. A. Whittaker, “What is software testing? And why is it so hard?” IEEE Software, vol. 17, no. 1, pp. 70–79, 2000. View at Publisher · View at Google Scholar · View at Scopus
  10. L. J. Osterweil, “Software processes are software too, revisited: an invited talk on the most influential paper of ICSE 9,” in Proceedings of the 19th IEEE International Conference on Software Engineering, pp. 540–548, Boston, Mass, USA, May 1997.
  11. ISO/IEC and ISO/IEC 29119-2, “Software Testing Standard—Activity Descriptions for Test Process Diagram,” 2008.
  12. O. Taipale, K. Smolander, and H. Kälviäinen, “Cost reduction and quality improvement in software testing,” in Proceedings of the 14th International Software Quality Management Conference (SQM '06), Southampton, UK, April 2006.
  13. O. Taipale, K. Smolander, and H. Kälviäinen, “Factors affecting software testing time schedule,” in Proceedings of the Australian Software Engineering Conference (ASWEC '06), pp. 283–291, Sydney, Australia, April 2006. View at Publisher · View at Google Scholar
  14. O. Taipale, K. Smolander, and H. Kälviäinen, “A survey on software testing,” in Proceedings of the 6th International SPICE Conference on Software Process Improvement and Capability dEtermination (SPICE '06), Luxembourg, May 2006.
  15. N. C. Dalkey, The Delphi Method: An Experimental Study of Group Opinion, RAND, Santa Monica, Calif, USA, 1969.
  16. S. P. Ng, T. Murnane, K. Reed, D. Grant, and T. Y. Chen, “A preliminary survey on software testing practices in Australia,” in Proceedings of the Australian Software Engineering Conference (ASWEC '04), pp. 116–125, Melbourne, Australia, April 2004.
  17. R. Torkar and S. Mankefors, “A survey on testing and reuse,” in Proceedings of IEEE International Conference on Software—Science, Technology and Engineering (SwSTE '03), Herzlia, Israel, November 2003.
  18. C. Ferreira and J. Cohen, “Agile systems development and stakeholder satisfaction: a South African empirical study,” in Proceedings of the Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists (SAICSIT '08), pp. 48–55, Wilderness, South Africa, October 2008. View at Publisher · View at Google Scholar
  19. J. Li, F. O. Bjørnson, R. Conradi, and V. B. Kampenes, “An empirical study of variations in COTS-based software development processes in the Norwegian IT industry,” Empirical Software Engineering, vol. 11, no. 3, pp. 433–461, 2006. View at Publisher · View at Google Scholar · View at Scopus
  20. W. Chen, J. Li, J. Ma, R. Conradi, J. Ji, and C. Liu, “An empirical study on software development with open source components in the Chinese software industry,” Software Process Improvement and Practice, vol. 13, no. 1, pp. 89–100, 2008. View at Publisher · View at Google Scholar · View at Scopus
  21. R. Dossani and N. Denny, “The Internet's role in offshored services: a case study of India,” ACM Transactions on Internet Technology, vol. 7, no. 3, 2007. View at Publisher · View at Google Scholar
  22. K. Y. Wong, “An exploratory study on knowledge management adoption in the Malaysian industry,” International Journal of Business Information Systems, vol. 3, no. 3, pp. 272–283, 2008. View at Publisher · View at Google Scholar · View at Scopus
  23. J. Bach, “Test automation snake oil,” in Proceedings of the 14th International Conference and Exposition on Testing Computer Software (TCS '99), Washington, DC, USA, June 1999.
  24. M. Fewster, Common Mistakes in Test Automation, Grove Consultants, 2001.
  25. A. Hartman, M. Katara, and A. Paradkar, “Domain specific approaches to software test automation,” in Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE '07), pp. 621–622, Dubrovnik, Croatia, September 2007. View at Publisher · View at Google Scholar
  26. C. Persson and N. Yilmaztürk, “Establishment of automated regression testing at ABB: industrial experience report on ‘avoiding the pitfalls’,” in Proceedings of the 19th International Conference on Automated Software Engineering (ASE '04), pp. 112–121, Linz, Austria, September 2004. View at Publisher · View at Google Scholar
  27. M. Auguston, J. B. Michael, and M.-T. Shing, “Test automation and safety assessment in rapid systems prototyping,” in Proceedings of the 16th IEEE International Workshop on Rapid System Prototyping (RSP '05), pp. 188–194, Montreal, Canada, June 2005. View at Publisher · View at Google Scholar
  28. A. Cavarra, J. Davies, T. Jeron, L. Mournier, A. Hartman, and S. Olvovsky, “Using UML for automatic test generation,” in Proceedings of the International Symposium on Software Testing and Analysis (ISSTA '02), Roma, Italy, July 2002.
  29. M. Vieira, J. Leduc, R. Subramanyan, and J. Kazmeier, “Automation of GUI testing using a model-driven approach,” in Proceedings of the International Workshop on Automation of Software Testing, pp. 9–14, Shanghai, China, May 2006. View at Publisher · View at Google Scholar
  30. Z. Xiaochun, Z. Bo, L. Juefeng, and G. Qiu, “A test automation solution on gui functional test,” in Proceedings of the 6th IEEE International Conference on Industrial Informatics (INDIN '08), pp. 1413–1418, Daejeon, Korea, July 2008. View at Publisher · View at Google Scholar
  31. D. Kreuer, “Applying test automation to type acceptance testing of telecom networks: a case study with customer participation,” in Proceedings of the 14th IEEE International Conference on Automated Software Engineering, pp. 216–223, Cocoa Beach, Fla, USA, October 1999. View at Publisher · View at Google Scholar
  32. W. D. Yu and G. Patil, “A workflow-based test automation framework for web based systems,” in Proceedings of the 12th IEEE Symposium on Computers and Communications (ISCC '07), pp. 333–339, Aveiro, Portugal, July 2007. View at Publisher · View at Google Scholar
  33. A. Bertolino, “Software testing research: achievements, challenges, dreams,” in Proceedings of the Future of Software Engineering (FoSE '07), pp. 85–103, Minneapolis, Minn, USA, May 2007. View at Publisher · View at Google Scholar
  34. M. Blackburn, R. Busser, and A. Nauman, “Why model-based test automation is different and what you should know to get started,” in Proceedings of the International Conference on Practical Software Quality, Braunschweig, Germany, September 2004.
  35. P. Santos-Neto, R. Resende, and C. Pádua, “Requirements for information systems model-based testing,” in Proceedings of the ACM Symposium on Applied Computing, pp. 1409–1415, Seoul, Korea, March 2007. View at Publisher · View at Google Scholar
  36. ISO/IEC and ISO/IEC 15504-1, “Information Technology—Process Assessment—Part 1: Concepts and Vocabulary,” 2002.
  37. K. M. Eisenhardt, “Building theories from case study research,” The Academy of Management Review, vol. 14, no. 4, pp. 532–550, 1989. View at Publisher · View at Google Scholar
  38. EU and European Commission, “The new SME definition: user guide and model declaration,” 2003.
  39. G. Paré and J. J. Elam, “Using case study research to build theories of IT implementation,” in Proceedings of the IFIP TC8 WG 8.2 International Conference on Information Systems and Qualitative Research, pp. 542–568, Chapman & Hall, Philadelphia, Pa, USA, May-June 1997.
  40. A. Strauss and J. Corbin, Basics of Qualitative Research: Grounded Theory Procedures and Techniques, SAGE, Newbury Park, Calif, USA, 1990.
  41. ATLAS.ti, The Knowledge Workbench, Scientific Software Development, 2005.
  42. M. B. Miles and A. M. Huberman, Qualitative Data Analysis, SAGE, Thousand Oaks, Calif, USA, 1994.
  43. C. B. Seaman, “Qualitative methods in empirical studies of software engineering,” IEEE Transactions on Software Engineering, vol. 25, no. 4, pp. 557–572, 1999. View at Google Scholar · View at Scopus
  44. C. Robson, Real World Research, Blackwell, Oxford, UK, 2nd edition, 2002.
  45. N. K. Denzin, The Research Act: A Theoretical Introduction to Sociological Methods, McGraw-Hill, New York, NY, USA, 1978.
  46. A. Fink and J. Kosecoff, How to Conduct Surveys: A Step-by-Step Guide, SAGE, Beverly Hills, Calif, USA, 1985.
  47. B. A. Kitchenham, S. L. Pfleeger, L. M. Pickard et al., “Preliminary guidelines for empirical research in software engineering,” IEEE Transactions on Software Engineering, vol. 28, no. 8, pp. 721–734, 2002. View at Publisher · View at Google Scholar · View at Scopus
  48. T. Dybå, “An instrument for measuring the key factors of success in software process improvement,” Empirical Software Engineering, vol. 5, no. 4, pp. 357–390, 2000. View at Publisher · View at Google Scholar · View at Scopus
  49. ISO/IEC and ISO/IEC 25010-2, “Software Engineering—Software product Quality Requirements and Evaluation (SQuaRE) Quality Model,” 2008.
  50. Y. Baruch, “Response rate in academic studies—a comparative analysis,” Human Relations, vol. 52, no. 4, pp. 421–438, 1999. View at Google Scholar · View at Scopus
  51. T. Koomen and M. Pol, Test Process Improvement: A Practical Step-by-Step Guide to Structured Testing, Addison-Wesley, Reading, Mass, USA, 1999.
  52. P. Kruchten, The Rational Unified Process: An Introduction, Addison-Wesley, Reading, Mass, USA, 2nd edition, 1998.
  53. K. Schwaber and M. Beedle, Agile Software Development with Scrum, Prentice-Hall, Upper Saddle River, NJ, USA, 2001.
  54. K. Beck, Extreme Programming Explained: Embrace Change, Addison-Wesley, Reading, Mass, USA, 2000.
  55. B. Glaser and A. L. Strauss, The Discovery of Grounded Theory: Strategies for Qualitative Research, Aldine, Chicago, Ill, USA, 1967.
  56. C. Kaner, “Improving the maintainability of automated test suites,” Software QA, vol. 4, no. 4, 1997. View at Google Scholar
  57. D. J. Mosley and B. A. Posey, Just Enough Software Test Automation, Prentice-Hall, Upper Saddle River, NJ, USA, 2002.
  58. D. Foray, Economics of Knowledge, MIT Press, Cambridge, Mass, USA, 2004.