Table of Contents Author Guidelines Submit a Manuscript
Advances in Human-Computer Interaction
Volume 2012, Article ID 369637, 13 pages
Research Article

Usability Testing for Serious Games: Making Informed Design Decisions with User Data

1Facultad de Informática, Universidad Complutense de Madrid, 28040 Madrid, Spain
2Laboratory of Computer Science, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA

Received 19 May 2012; Revised 9 October 2012; Accepted 11 October 2012

Academic Editor: Kiju Lee

Copyright © 2012 Pablo Moreno-Ger et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. S. de Freitas and M. Oliver, “How can exploratory learning with games and simulations within the curriculum be most effectively evaluated?” Computers and Education, vol. 46, no. 3, pp. 249–264, 2006. View at Publisher · View at Google Scholar · View at Scopus
  2. J. Nielsen, “Heuristic evaluation,” in Usability Inspection Methods, J. Nielsen and R. L. Mack, Eds., vol. 17, pp. 25–62, John Wiley & Sons, 1994. View at Google Scholar
  3. M. Kessner, J. Wood, R. F. Dillon, and R. L. West, “On the reliability of usability testing,” in Proceedings of the Extended Abstracts on Human Factors in Computing Systems (CHI '01), p. 97, 2001.
  4. M. Macleod and R. Rengger, “The development of DRUM: a software tool for video-assisted usability evaluation,” in Proceedings of the 5th International Conference on Human-Computer Interaction (HCI '93), pp. 293–309, August 1993.
  5. R. J. Pagulayan, K. Keeker, D. Wixon, R. L. Romero, and T. Fuller, “User-centered design in games,” in Design, J. A. Jacko and A. Sears, Eds., vol. 28, pp. 883–906, Lawrence Erlbaum Associates, 2003. View at Google Scholar
  6. R. Koster, Theory of Fun for Game Design, Paraglyph, Scottsdale, Ariz, USA, 2004.
  7. E. Ju and C. Wagner, “Personal computer adventure games: their structure, principles, and applicability for training,” Data Base for Advances in Information Systems, vol. 28, no. 2, pp. 78–92, 1997. View at Google Scholar · View at Scopus
  8. International Organization For Standardization, “ISO, 9241-11: guidance on usability,” Ergonomic requirements for office work with visual display terminals, 1998,
  9. A. R. Cooper, The Inmates are Running the Asylum: Why High Tech Products Drive us Crazy and How to Restore the Sanity, Macmillan Publishing, Indianapolis, Ind, USA, 1999.
  10. J. Nielsen and R. Molich, “Heuristic evaluation of user interfaces,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Empowering People (CHI '90), pp. 249–256, 1990.
  11. J. Brooke, “SUS: a ‘quick and dirty’usability scale’,” in Usability Evaluation in Industry, P. W. Jordan, B. Thomas, B. A. Weerdmeester, and I. L. McClelland, Eds., pp. 189–194, Taylor & Francis, London, UK, 1996. View at Google Scholar
  12. J. Kirakowski and M. Corbett, “SUMI: the software usability measurement inventory,” British Journal of Educational Technology, vol. 24, no. 3, pp. 210–212, 1993. View at Google Scholar
  13. B. D. Harper and K. L. Norman, “Improving user satisfaction: the questionnaire for user interaction satisfaction version 5. 5,” in Proceedings of the 1st Annual Mid-Atlantic Human Factors Conference, pp. 224–228, 1993.
  14. H. W. Jung, S. G. Kim, and C. S. Chung, “Measuring software product quality: a survey of ISO/IEC 9126,” IEEE Software, vol. 21, no. 5, pp. 88–92, 2004. View at Publisher · View at Google Scholar · View at Scopus
  15. I. Wechsung and A. B. Naumann, “Evaluation methods for multimodal systems: a comparison of standardized usability questionnaires,” in Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems (PIT '08), vol. 5078 of Lecture Notes in Computer Science, pp. 276–284, 2008.
  16. R. L. Boring, D. I. Gertman, J. C. Joe, and J. L. Marble, “Proof of concept for A human reliability analysis method for heuristic usability evaluation of software,” in Proceedings of the 49th Annual Meeting of the Human Factors and Ergonomics Society (HFES '05), pp. 676–680, Orlando, Fla, USA, September 2005. View at Scopus
  17. E. L. C. Law and E. T. Hvannberg, “Analysis of strategies for improving and estimating the effectiveness of heuristic evaluation,” in Proceedings of the 3rd Nordic Conference on Human-Computer Interaction (NordiCHI '04), pp. 241–250, Tampere, Finland, October 2004. View at Publisher · View at Google Scholar · View at Scopus
  18. P. Moreno-Ger, D. Burgos, and J. Torrente, “Digital games in elearning environments: current uses and emerging trends,” Simulation & Gaming, vol. 40, no. 5, pp. 669–687, 2009. View at Google Scholar
  19. J. Kirriemuir and A. McFarlane, Literature Review in Games and Learning, NESTA Futurelab, Bristol, UK, 2004.
  20. R. Van Eck, “Digital game-based learning: it’s not just the digital natives who are restless,” EDUCAUSE Review, vol. 41, no. 2, pp. 16–30, 2006. View at Google Scholar
  21. J. P. Gee, Good Videogames and Good Learning: Collected Essays on Video Games, Peter Lang Publishing, New York, NY, USA, 2007.
  22. V. J. Shute, I. Masduki, and O. Donmez, “Conceptual framework for modeling, assessing, and supporting competencies within game environments,” Technology, Instruction, Cognition, and Learning, vol. 8, no. 2, pp. 137–161, 2010. View at Google Scholar
  23. C. S. Loh, “Designing online games assessment as information trails,” in Games and Simulations in Online Learning: Research and Development Frameworks, D. Gibson, C. Aldrich, and M. Prensky, Eds., pp. 323–348, Information Science Publishing, Hershey, Pa, USA, 2007. View at Google Scholar
  24. K. Squire, “Changing the game: what happens when video games enter the classroom,” Innovate, vol. 1, no. 6, 2005. View at Google Scholar
  25. M. P. Eladhari and E. M. I. Ollila, “Design for research results: experimental prototyping and play testing,” Simulation & Gaming, vol. 43, no. 3, pp. 391–412, 2012. View at Google Scholar
  26. E. Ollila, Using Prototyping and Evaluation Methods in Iterative Design of Innovative Mobile Games, Tampere University of Technology, Tampere, Finland, 2009.
  27. J. A. Garcia Marin, E. Lawrence, K. Felix Navarro, and C. Sax, “Heuristic Evaluation for Interactive Games within Elderly Users,” in Proceedings of the 3rd International Conference on eHealth, Telemedicine, and Social Medicine (eTELEMED '11), pp. 130–133, 2011.
  28. D. Pinelle and N. Wong, “Heuristic evaluation of games,” in Game Usability Advice from the Experts for Advancing the Player Experience, K. Isbister and N. Schaffer, Eds., pp. 79–89, ACM Press, 2008. View at Google Scholar
  29. W. Ijsselsteijn, Y. De Kort, K. Poels, A. Jurgelionis, and F. Bellotti, “Characterising and measuring user experiences in digital games,” in Proceedings of the Avances in Computer Entertainment (ACE '07), June 2007.
  30. K. M. Gilleade and A. Dix, “Using frustration in the design of adaptive videogames,” in Proceedings of the ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (ACE '04), pp. 228–232, Singapore, June 2005. View at Scopus
  31. G. Sim, S. MacFarlane, and J. Read, “All work and no play: measuring fun, usability, and learning in software for children,” Computers and Education, vol. 46, no. 3, pp. 235–248, 2006. View at Publisher · View at Google Scholar · View at Scopus
  32. G. R. White, P. Mirza-Babaei, G. McAllister, and J. Good, “Weak inter-rater reliability in heuristic evaluation of video games,” in Proceedings of the 29th Annual CHI Conference on Human Factors in Computing Systems (CHI '11), pp. 1441–1446, May 2011. View at Publisher · View at Google Scholar · View at Scopus
  33. R. A. Virzi, “Refining the test phase of usability evaluation: how many subjects is enough?” Human Factors, vol. 34, no. 4, pp. 457–468, 1992. View at Google Scholar · View at Scopus
  34. J. Nielsen and T. K. Landauer, “Mathematical model of the finding of usability problems,” in Proceedings of the Conference on Human Factors in Computing Systems (INTERACT '93) and (CHI '93), pp. 206–213, April 1993. View at Scopus