Table of Contents Author Guidelines Submit a Manuscript
Advances in Human-Computer Interaction
Volume 2008, Article ID 874563, 19 pages
http://dx.doi.org/10.1155/2008/874563
Research Article

Embodiment, Multimodality, and Composition: Convergent Themes across HCI and Education for Mixed-Reality Learning Environments

1Arts, Media and Engineering, Arizona State University, Tempe, AZ 85281, USA
2Department of Electrical Engineering, Arizona State University, Tempe, AZ 85281, USA
3School of Educational Innovation and Teacher Preparation, Arizona State University, Tempe, AZ 85281, USA
4School of Art, Arizona State University, Tempe, AZ 85281, USA
5School of Computing and Informatics, Arizona State University, Tempe, AZ 85281, USA
6Department of Psychology, Arizona State University, Tempe, AZ 85281, USA

Received 6 October 2007; Revised 27 July 2008; Accepted 14 October 2008

Academic Editor: Adrian Cheok

Copyright © 2008 David Birchfield et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. J. D. Bransford, A. L. Brown, and R. R. Cocking, Eds., How People Learn: Brain, Mind, Experience, and School, National Academy Press, Washington, DC, USA, 2000.
  2. E. Hutchins, “Material anchors for conceptual blends,” in Proceedings of the Symposium on Conceptual Blending, Odense, Denmark, 2002.
  3. D. Hestenes, “Notes for a modeling theory of science cognition and instruction,” in Proceedings of the GIREP Conrerence on Modeling in Physics and Physics Education (GIREP '06), p. 27, Amsterdam, The Netherlands, August 2006.
  4. G. Fauconnier and M. Turner, The Way We Think: Conceptual Blending and the Mind's Hidden Complexities, Basic Books, New York, NY, USA, 2002.
  5. L. W. Barsalou, “Grounded cognition,” Annual Review of Psychology, vol. 59, no. 1, pp. 617–645, 2008. View at Publisher · View at Google Scholar
  6. M. C. Megowan-Romanowicz, Framing discourse for optimal learning in science and mathematics, Ph.D. dissertation, College of Education, Division of Curriculum and Instruction, Arizona State University, Tempe, Ariz, USA, 2007.
  7. A. M. Glenberg, T. Gutierrez, J. R. Levin, S. Japuntich, and M. P. Kaschak, “Activity and imagined activity can enhance young children's reading comprehension,” Journal of Educational Psychology, vol. 96, no. 3, pp. 424–436, 2004. View at Publisher · View at Google Scholar
  8. P. Dourish, Where the Action is: The Foundations of Embodied Interaction, MIT Press, Cambridge, Mass, USA, 2001.
  9. P. Dourish, “What we talk about when we talk about context,” Journal of Personal and Ubiquitous Computing, vol. 8, no. 1, pp. 19–30, 2004. View at Publisher · View at Google Scholar
  10. J. Rambusch and T. Ziemke, “The role of embodiment in situated learning,” in Proceedings of the 27th Annual Conference of the Cognitive Science Society, pp. 1803–1808, Stresa, Italy, July 2005.
  11. L. Suchman, Plans and Situated Actions: The Problem of Human-Machine Communication, Cambridge University Press, New York, NY, USA, 1987.
  12. F. Varela, E. Thompson, and E. Rosch, The Embodied Mind: Cognitive Science and Human Experience, MIT Press, Cambridge, Mass, USA, 1991.
  13. E. Hutchins, Cognition in the Wild, vol. 1, MIT Press, Cambridge, Mass, USA, 1995.
  14. M. Wilson, “Six views of embodied cognition,” Psychonomic Bulletin and Review, vol. 9, no. 4, pp. 625–636, 2002. View at Google Scholar
  15. J. J. Gibson, The Ecological Approach to Visual Perception, Houghton Mifflin, Boston, Mass, USA, 1979.
  16. D. Norman, The Psychology of Everyday Things, Basic Books, New York, NY, USA, 1988.
  17. R. N. Shepard, “Perceptual-cognitive universals as reflections of the world,” Behavioral and Brain Sciences, vol. 24, no. 4, pp. 581–601, 2001. View at Publisher · View at Google Scholar
  18. A. M. Glenberg, D. Havas, R. Becker, and M. Rinck, “Grounding language in bodily states: the case for emotion,” in The Grounding of Cognition: The Role of Perception and Action in Memory, Language, and Thinking, R. Zwann and D. Pecher, Eds., Cambridge University Press, Cambridge, UK, 2005. View at Google Scholar
  19. P. I. Galperin, “Stage-by-stage formation as a method of psychological investigation,” Journal of Russian and East European Psychology, vol. 30, no. 4, pp. 60–80, 1992. View at Google Scholar
  20. I. Harel, Children Designers, Ablex, Norwood, NJ, USA, 1991.
  21. S. Papert, Mindstorms: Children, Computers and Powerful Ideas, Basic Books, New York, NY, USA, 1980.
  22. S. Papert, The Children's Machine, Basic Books, New York, NY, USA, 1993.
  23. M. Resnick, “Technologies for lifelong kindergarten,” Educational Technology Research and Development, vol. 46, no. 4, pp. 43–55, 1998. View at Publisher · View at Google Scholar
  24. S. Papert, “Situating constructionism,” in Constructionism, pp. 1–12, Ablex, Norwood, NJ, USA, 1991. View at Google Scholar
  25. K. Peppler and Y. B. Kafai, “From SuperGoo to Scratch: exploring creative digital media production in informal learning,” Learning, Media and Technology, vol. 32, no. 2, pp. 149–166, 2007. View at Publisher · View at Google Scholar
  26. H. Ishii and B. Ullmer, “Tangible bits: towards seamless interfaces between people, bits and atoms,” in Proceedings of the Conference on Human Factors in Computing Systems (CHI '97), pp. 234–241, Atlanta, Ga, USA, March 1997.
  27. H. Ishii, “Tangible User Interfaces,” 2007, http://tangible.media.mit.edu/.
  28. C. Cruz-Neira, D. J. Sandin, and T. A. DeFanti, “Surround-screen projection-based virtual reality: the design and implementation of the CAVE,” in Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, pp. 135–142, Anaheim, Calif, USA, August 1993. View at Publisher · View at Google Scholar
  29. W. Swartout, R. Hill, J. Gratch et al., “Toward the holodeck: integrating graphics, sound, character and story,” in Proceedings of the 5th International Conference on Autonomous Agents, pp. 409–416, Montreal, Canada, May-June 2001.
  30. G. Weinberg, “The Musical Playpen: an immersive digital musical instrument,” Personal and Ubiquitous Computing, vol. 3, no. 3, pp. 132–136, 1999. View at Publisher · View at Google Scholar
  31. A. F. Bobick, S. S. Intille, J. W. Davis et al., “The kidsRoom: a perceptually-based interactive and immersive story environment,” Presence: Teleoperators and Virtual Environments, vol. 8, no. 4, pp. 369–393, 1999. View at Publisher · View at Google Scholar
  32. D. Cavallo, A. Sipitakiat, A. Basu et al., “RoBallet: exploring learning through expression in the arts through constructing in a technologically immersive environment,” in Proceedings of the 6th International Conference on Learning Sciences, pp. 105–112, Santa Monica, Calif, USA, June 2004.
  33. C. Dede and D. J. Ketelhut, “Designing for motivation and usability in a museum-based multi-user virtual environment,” in Proceedings of the American Educational Research Association Conference, Chicago, Ill, USA, April 2003.
  34. C. Dede, D. J. Ketelhut, and K. Ruess, “Motivation, usability and learning outcomes in a prototype museum-based multi-user virtual environment,” in Proceedings of the 5th International Conference of the Learning Sciences, Seattle, Wash, USA, October 2002.
  35. D. J. Ketelhut, “The impact of student self-efficacy on scientific inquiry skills: an exploratory investigation in River City, a multi-user virtual environment,” Journal of Science Education and Technology, vol. 16, no. 1, pp. 99–111, 2007. View at Publisher · View at Google Scholar
  36. B. C. Nelson, D. J. Ketelhut, J. Clarke, E. Dieterle, C. Dede, and B. Erlandson, “Robust design strategies for scaling educational innovations: the River City MUVE case study,” in The Design and Use of Simulation Computer Games in Education, pp. 209–231, Sense Press, Rotterdam, The Netherlands, 2007. View at Google Scholar
  37. H. Gardner, Frames of Mind: The Theory of Multiple Intelligences, Basic Books, New York, NY, USA, 1993.
  38. R. Jackendoff, “The architecture of the linguistic spatial interface,” in Language and Space, P. Bloom, M. A. Peterson, L. Nadel, and M. F. Garrett, Eds., pp. 1–30, MIT Press, Cambridge, Mass, USA, 1996. View at Google Scholar
  39. D. Cunningham, T. Duffy, and R. Knuth, “Textbook of the future,” in Hypertext: A Psychological Perspective, C. McKnight, Ed., pp. 19–50, Ellis Horwood, London, UK, 1993. View at Google Scholar
  40. W.-M. Roth and M. K. McGinn, “Inscriptions: toward a theory of representing as social practice,” Review of Educational Research, vol. 68, no. 1, pp. 35–59, 1998. View at Google Scholar
  41. G. Burt, “Media effectiveness, essentiality, and amount of study: a mathematical model,” British Journal of Educational Technology, vol. 37, no. 1, pp. 121–130, 2006. View at Publisher · View at Google Scholar
  42. C. R. Wren, S. Basu, F. Sparacino, and A. Pentland, “Combining audio and video in perceptive spaces,” in Proceedings of the 1st International Workshop on Managing Interactions in Smart Environments (MANSE '99), pp. 44–55, Dublin, Ireland, December 1999.
  43. C. Bahn, T. Hahn, and D. Trueman, “Physicality and feedback: a focus on the body in the performance of electronic music,” in Proceedings of the International Computer Music Conference (ICMC '01), San Francisco, Calif, USA, November 2001.
  44. A. Hunt, M. Wanderley, and M. Paradis, “The importance of parameter mapping in electronic instrument design,” in Proceedings of the International Conference on New Interfaces for Musical Expression (NIME '02), Dublin, Ireland, May 2002.
  45. D. Wessel and M. Wright, “Problems and prospects for intimate musical control of computers,” in Proceedings of the International Conference on New Interfaces for Musical Expression (NIME '01), pp. 1–4, Seattle, Wash, USA, April 2001.
  46. P. Rheingans and C. Landreth, “Perceptual principles for effective visualizations,” in Perceptual Issues in Visualization, pp. 59–73, Springer, Berlin, Germany, 1995. View at Google Scholar
  47. L. Muller and E. Edmonds, “Living laboratories: making and curating interactive art,” in Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques, Boston, Mass, USA, July-August 2006. View at Publisher · View at Google Scholar
  48. EU Community Report, Information Society Technologies (IST) Programme, “A multisensory environment design for an interface between autistic and typical expressiveness,” 2004. View at Google Scholar
  49. N. Parés, A. Carreras, J. Durany et al., “MEDIATE: an interactive multisensory environment for children with severe autism and no verbal communication,” in Proceedings of the 3rd International Workshop on Virtual Rehabilitation (IWVR '04), Lausanne, Switzerland, Septembe 2004.
  50. N. Parés, P. Masri, G. van Wolferen, and C. Creed, “Achieving dialogue with children with severe autism in an adaptive multisensory interaction: the “MEDIATE” project,” IEEE Transactions on Visualization and Computer Graphics, vol. 11, no. 6, pp. 734–742, 2005. View at Publisher · View at Google Scholar
  51. L. S. Vygotsky, Mind in Society: The Development of Higher Psychological Processes, The Harvard University Press, Cambridge, Mass, USA, 1978.
  52. M. Dougiamas, “A journey into Constructivism,” November 1998. View at Google Scholar
  53. P. Cobb, “Constructivism in math and science education,” Educational Researcher, vol. 23, no. 7, p. 4, 1994. View at Google Scholar
  54. L. S. Vygotsky, Thought and Language, MIT Press, Cambridge, Mass, USA, 1962.
  55. R. Lesh and H. Doerr, Beyond Constructivism: Models and Modeling Perspectives on Mathematics Problem Solving, Learning, and Teaching, Lawrence Erlbaum Associates, Mahwah, NJ, USA, 2003.
  56. J. S. Brown, A. Collins, and P. Duguid, “Situated cognition and the culture of learning,” Educational Researcher, vol. 18, no. 1, pp. 32–42, 1989. View at Google Scholar
  57. J. A. Middleton and Z. Toluk, “First steps in the development of an adaptive theory of motivation,” Educational Psychologist, vol. 34, no. 2, pp. 99–112, 1999. View at Publisher · View at Google Scholar
  58. M. Resnick, “Computer as paintbrush: technology, play, and the creative society,” in Play = Learning: How Play Motivates and Enhances Children's Cognitive and Social-Emotional Growth, D. Singer, R. Golikoff, and K. Hirsh-Pasek, Eds., Oxford University Press, Cambridge, UK, 2006. View at Google Scholar
  59. K. Ryokai, S. Marti, and H. Ishii, “I/O brush: drawing with everyday objects as ink,” in Proceedings of the Conference on Human Factors in Computing Systems (CHI '04), pp. 303–310, Vienna, Austria, April 2004.
  60. M. Resnick, “All I really need to know (about creative thinking) I learned (by studying how chidren learn) in kindergarten,” in Proceedings of the 6th ACM SIGCHI Conference on Creativity and Cognition, pp. 1–6, Washington, DC, USA, June 2007. View at Publisher · View at Google Scholar
  61. Y. B. Kafai, K. Peppler, and G. Chin, “High tech programmers in low income communities: creating a computer culture in a community technology center,” in Proceedings of the 3rd International Conference on Communities and Technologies, C. Steinfeld, B. Pentland, M. Achkermann, and N. Contractor, Eds., pp. 545–562, East Lansing, Mich, USA, June 2007.
  62. K. Salen and E. Zimmerman, Rules of Play: Game Design Fundamentals, MIT Press, Cambridge, Mass, USA, 2004.
  63. J. P. Gee, What Video Games have to Teach Us about Learning and Literacy, Palgrave Macmillan, New York, NY, USA, 2003.
  64. H. Jenkins, K. Clinton, R. Purushotma, A. Robison, and M. Weigel, Confronting the Challenges of Participatory Culture: Media Education for the 21st Century: MacArthur Foundation, 2006.
  65. SmartUs, “SmartUs: Games in Motion,” 2008, http://www.smartus.com/.
  66. D. Birchfield, T. Ciufo, and G. Minyard, “SMALLab: a mediated platform for education,” in Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques, p. 33, Boston, Mass, USA, July-August 2006. View at Publisher · View at Google Scholar
  67. A. Shirai, E. Geslin, and S. Richir, “WiiMedia: motion analysis methods and applications using a consumer video game controller,” in Proceedings of the ACM SIGGRAPH Symposium on Video Games, pp. 133–140, San Diego, Calif, USA, August 2007. View at Publisher · View at Google Scholar
  68. M. Brain, “How the Wii Works,” 2007, http://electronics.howstuffworks.com/wii.htm.
  69. H. Jin, G. Qian, and S. Rajko, “Real-time multi-view 3D object tracking in cluttered scenes,” in Proceedings of the 2nd International Symposium on Visual Computing (ISVC '06), pp. 647–656, Lake Tahoe, Nev, USA, November 2006. View at Publisher · View at Google Scholar
  70. K. Tu, H. Thornburg, E. Campana, D. Birchfield, M. Fulmer, and A. Spanias, “Interaction and reflection via 3D path shape qualities in a mediated constructive learning environment,” in Proceedings of the International Workshop on Educational Multimedia and Multimedia Education (EMME '07), pp. 37–46, Augsburg, Germany, September 2007. View at Publisher · View at Google Scholar
  71. K. Tu, H. Thornburg, M. Fulmer, and A. Spanias, “Tracking the path shape qualities of human motion,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '07), vol. 2, pp. 781–784, Honolulu, Hawaii, USA, April 2007. View at Publisher · View at Google Scholar
  72. I. Dolgov, D. Birchfield, M. McBeath, H. Thornburg, and C. Todd, “Perception of approaching and retreating floor-projected shapes in a large, immersive, multimedia learning environment,” Journal of Perceptual and Motor Skills. In press.
  73. A. Cuthbertson, S. Hatton, G. Minyard, H. Piver, C. Todd, and D. Birchfield, “Mediated education in a creative arts context: research and practice at Whittier Elementary School,” in Proceedings of the 6th International Conference on Interaction Design and Children (IDC '07), pp. 65–72, Aalborg, Danmark, June 2007. View at Publisher · View at Google Scholar
  74. S. Hatton, D. Birchfield, and M. C. Megowan-Romanowicz, “Learning metaphor through mixed-reality game design and game play,” in Proceedings of the ACM SIGGRAPH Symposium on Video Games (Sandbox '08), pp. 67–74, Los Angeles, Calif, USA, August 2008. View at Publisher · View at Google Scholar
  75. D. Birchfield, B. Mechtley, S. Hatton, and H. Thornburg, “Mixed-reality learning in the art museum context,” in Proceedings of ACM SIG Multimedia, Vancouver, Canada, October-November 2008.
  76. D. Chen and W. Stroup, “General system theory: toward a conceptual framework for science and technology education for all,” Journal of Science Education and Technology, vol. 2, no. 3, pp. 447–459, 1993. View at Publisher · View at Google Scholar
  77. F. Lutgens, E. Tarbuck, and D. Tasa, Foundations of Earth Science, Prentice Hall, Upper Saddle River, NJ, USA, 4th edition, 2004.
  78. Arizona Department of Education, “Science Standard Articulated by Grade Level Strand 6: Earth and Space Science,” 2005, http://www.ade.az.gov/standards/science/downloads/strand6.pdf.
  79. D. Hestenes, “Modeling methodology for physics teachers,” in Proceedings of the International Conference on Undergraduate Physics Education (ICUPE '96), College Park, Md, USA, July-August 1996.
  80. D. Hestenes, “Modeling games in the newtonian world,” American Journal of Physics, vol. 60, no. 8, pp. 732–748, 1992. View at Publisher · View at Google Scholar
  81. R. R. Hake, “Interactive-engagement vs traditional methods: a six-thousand-student survey of mechanics test data for introductory physics courses,” American Journal of Physics, vol. 66, no. 1, pp. 64–74, 1998. View at Publisher · View at Google Scholar