Abstract
Natural User Interface (NUI) is a new approach that has become increasingly popular in Human-Computer Interaction (HCI). The use of this technology is widely used in almost all sectors, including the field of education. In recent years, there are a lot of educational games using NUI technology in the market such as Kinect game. Kinect is a sensor that can recognize body movements, postures, and voices in three dimensions. It enables users to control and interact with game without the need of using game controller. However, the contents of most existing Kinect games do not follow the standard curriculum in classroom, thus making it do not fully achieve the learning objectives. Hence, this research proposes a design model as a guideline in designing educational game using NUI. A prototype has been developed as one of the objectives in this study. The prototype is based on proposed model to ensure and assess the effectiveness of the model. The outcomes of this study conclude that the proposed model contributed to the design method for the development of the educational game using NUI. Furthermore, evaluation results of the prototype show a good response from participant and in line with the standard curriculum.
1. Introduction
Serious game has strong potential for learning. It can be used as an innovative tool for constructive teaching approaches in classrooms. However, students usually interact with computer using mouse and keyboard while playing the game. This action gives a negative impact to students while sitting too long staring at the computer screen [1]. A good learning should involve all the senses and limbs to act and react. The appearance of Natural User Interface (NUI) gives a new way on interacting with computer. NUI refers to sensory inputs such as touch, speech, and gesture but goes much further to describe computing. Computing is described as intelligent and contextually aware, with the ability to recognize a person’s face, environment and intent, and also emotions and relationships.
Several technologies use NUI for their interfaces such as Microsoft Kinect, Microsoft PixelSens, Leap Motion, and Intel RealSense. For the game context, Microsoft Kinect is most popular than the others. Kinect described as a revolution in the making, because it provides a brand new type of interaction with computers [2]. Motion sensing input device by Microsoft for the Xbox 360 video game console can capture, track, and decipher body movements such as gestures and voice. In other words, users are not by bound neither keyboards, mouse, nor joysticks and thus have intuitive and virtual experiences with digital contents. In recent years, there are many educational games using Kinect technology making an additional option in the learning market. However, the contents of most Kinect games do not follow the standard curriculum for classroom, so it cannot fully achieve the learning objectives. Any media and resources in classroom are more effective when integrated into the curriculum [3].
Kinect technology has potential to enhance classroom interactions, but it largely depends on the development of Kinect software [4]. The effectiveness of the Kinect game depends on the content and how the game is developed. Simply adding a curriculum to the game does not mean they were integrate with it. However, the implementation of Kinect in the classroom has pedagogical constraint that must be considered such as the difficulties in shifting to kinesthetic pedagogical practices and limited understanding of its effect [4]. Therefore, this paper proposes design model guideline for development of NUI game-based on learning.
2. Objectives
The main objectives of this research are to (1) develop a design model for NUI game-based on learning and demonstrate the model by designing a prototype and (2) evaluate the prototype to ensure it is in line with the standard curriculum. The scope of this research incorporates the following: (1) the subject content selected for this prototype is Science subject for primary school year 3; one game under subtitle “classification” was created and (2) the testing of prototype is done by fifteen students to assess the effectiveness of the proposed model.
3. Literature Review
The key in education is how games are used. Simply adding games to a curriculum does not mean they are integrated with it. The students should be involved, through technology, in the learning process in order to (a) be engaged with the theories, (b) acquire knowledge through autonomous and discovery learning, (c) cultivate thinking skills, (d) learn how to learn (metacognition), (e) interact and communicate, and (f) operate as active producers of knowledge [5].
There are several games that use NUI in education, such as Kids Magic Learning. This Kinect game was designed for early childhood education and contain six different games. Each game was accompanied by some specific courses such as physical coordination, instant memory, ability in mathematics, and ability of memorizing vocabulary and speaking including sense of space [6]. Figure 1 shows one example of the games.
Unfortunately, the existing games mostly only be used in teaching basic skills. Research was done to analyze Kinect game existence in the market [7]. This research attempts to categorize the most popular Kinect games. The result was the games that are teaching basic skills such as physical, emotional, cognitive, and social development. Table 1 shows the result of the research. Moreover, the proposed categorization consists an initial approach to these games which are considered necessary in order to conduct future research to investigate what is the impact of Kinect games in students’ life exactly.
4. Design Model
One of the keys on designing frameworks or models is to identify the core elements and patterns that define the complex structure of what makes a successful video game [8], where the player’s main objective is having fun but the real goal of the activities is still the learning experience.
4.1. Serious Game Elements
According to Annetta [9] in “The I’6s Model,” there are six elements of serious game. These six elements appear in an order of magnitude: identity, immersion, interactivity, increasing complexity, informed teaching, and instructional. Figure 2 illustrates a nest model of the six elements for developing serious game.
Identity is ability to capture the player’s mind and trick him or her into believing that he or she is in the game environment. In many modern video games, the player’s identity is represented through a unique character called an avatar.
Immersion is where the players have a higher sense of presence through individual identity, engaged in the content and intrinsically motivated to succeed in the challenge of the game’s goal.
Interactivity is where the games allow players to be social communicators, whether it is with other players in a multiplayer environment or with the machine. High interactivity refers to a situation in which all the component parts are interconnected and there is feedback between each other.
Increasing complexity is where the games have multiple levels. Level provides a platform to increase the complexity of the concept and content of a game.
Informed teaching is the feedback and embedded assessments within games. The game should contain a scoring mechanism used to determine the player progress.
Instructional is where the game can be used as instruction media. Learning is the goal of any educational endeavor. For games to be instructional, they need to have all of the aforementioned components in addition to the following concepts.
4.2. NUI Element
The NUI is defined by three elements [10]:(i)Enjoyable.(ii)Leading to skilled practice.(iii)Appropriate to context.
These elements are joined by “and.” NUI must have all of these elements in varying degrees. Games are enjoyable, but unfortunately many of them have nothing to do with NUI. Many training systems (e.g., Mavis Beacon teaches typing) lead to skilled practice but are not NUIs. An ATM is appropriate to context but is not NUI.
4.3. Curriculum Integration
The architecture of succeed curriculum integration is when curriculum components are connected and related in a meaningful way to learners [11]. This can be achieved through three fundamental steps (see Figure 3).
Step 1 (plot a course based on component of present curriculum that can benefit from game-enhanced approaches). This approach involves the dissection of curriculum into components, for example, if we took an introductory statistic course and dissect it into (1) statistical theory, (2) worked examples, and (3) final component that applies worked example to real world problem. This components dissection further lends to reinforcement of the theoretical underpinnings of the subject matter, as the learner now has a tool that provides a causal approach to the content and its outcomes.
Step 2 (create an environment of learning flexibility). Learning flexibility removes in-built perceptions, attitude, and belief structures akin with traditional curriculum. So it will change the learners from those who need to be tested to learners whose desire to understand through the causality associated with games play.
Step 3 (focus initially on subject matter that can be efficiently measured and then expanded). In this step, the focus is on subject matter that can be efficiently measured and then expanded for game-enhanced learning that is critical. The other meaning of this is to apply a systematic approach for game integration by the following: (1)Removing learning barrier: the virtual wall between learners and academic needs to be sufficiently removed to allow student to perceive their teachers more as coaches than academic stalwarts.(2)Enable the subject matter: trying something new such as game that enhanced curriculum should include some “Easter egg,” that is, hidden messages or clues, which support student to take new direction.(3)Direct subject matter flow: measure initial subject matter through strategically mapping subject content into specific elements of the game, to enhance the curriculum.
4.4. Pedagogy
Pedagogy is the discipline that deals with the theory and practice of education. It also concerns at the study and practice of the best ways to teach. Successful game is a combination of potentially adaptive structures (such as rules) and the delivery time of information. Both of these are relevant when teaching; hence the institute of education argues that using game design theory can inform virtual world pedagogy [12]. Those who adopt this view frequently situate the learning process within a framework of pedagogical theories such as Kolb’s experiential learning.
As a theoretical model, experiential learning was first defined by Kolb as “the process whereby knowledge is created through the transformation of experience.” According to Kolb’s model (see Figure 4), learning is a cyclical process which consists of four main stages. Beginning with a concrete experience, the learner makes observations and reflects on this experience. Based on these reflection, he or she draws conclusions and generalizes on how their new knowledge can be used in other scenarios. Finally, the learner tests these generalizations and hypotheses through experimentations and further experiences [13].
In designing serious games, there are three factors involved, that is, theory; contents; and game design [14]. The theory included pedagogy, cognition learning, and behavioral learning. The content is the course that will be the game’s content. Game design includes Artificial Intelligence (AI), Human-Computer Interaction (HCI), a User Interface (UI), and programming.
In order to use game as a learning tool, we need to consider three factors that had been mentioned above. Figure 5 describes the relationship between the three factors in the development of serious games. Educational games are identical with serious game. Therefore, the relationship between the components of the proposed model in this study can be seen in Figure 6.
NUI educational game should provide a quality content for the target domain that is in line with curriculum and based on pedagogy as a discipline of education. After that, during the development stages, the involvement of six serious game elements and three elements of NUI needs to be ensured.
5. Prototype
The prototype was implemented in Unity platform, using C Sharp programming language and Microsoft Kinect as NUI technology platform. It was developed based on the proposed model and the aims are to ensure the effectiveness of the model. Before starting to develop the game, meeting and interview with Science teacher were done to collect the data about curriculum needs.
The fundamental steps of curriculum integration applied on the data to select the content of the game. The selection was based on analysis about students’ needs and abilities on the subject. The analysis was conducted by interviewing the teachers. The results of the interview were recorded and classified according to students’ interest in learning each title and then assessed in percent. A high percentage indicates high interest in learning the subject. The low percentage is selected as the content of prototype. The results are shown in Table 2.
The next step, selected content must be appropriate to the pedagogical approach. The content of the game must deal with pedagogy to be relevant when used as a teaching tool in classroom. The appropriate methods in teaching and learning Science are a process-oriented approach. This is consistent with Kolb Theory that used experiential learning for its methods. In this prototype, we select subtittle “classification” as a content of the game. Classification is the arrangement of objects, ideas, or information into groups and the members which have one or more characteristics in common. The format of this game is to classify objects based on their characteristics. This format is commonly and widely used in digital games such as puzzle games that use drag-and-drop to collect objects. However, in this study the game structure uses gestures provided by NUI. Users can classify objects by moving their bodies; it will make the game more interesting. The gestures used included raised right hand, lowered right hand, raised left hand, and lower left hand as shown in Figure 7.
This four gestures control the movement of the bridge. The object to be classified across the bridge to get to the classification or characteristics that are across the bridge. Players need to use the gestures to make sure the objects go to the correct classification. Game sketches is shown in Figure 8. Players have to raise their right hand to raise the right bridge and to make it lower, players have to lower their hand. As well as for the left bridge, players can control it using their left hand. This format was designed to ensure that students have the ability to classify object properly and quickly.
The next step was designing the game. Six serious game elements and three elements of NUI should be applied in this step. Figure 9 shows the interface of the game. We developed colorful interface and added sound effects and animation to make the game look more interesting for children.
All elements that have been studied previously should be considered and included into the gameplay for making the game effective and can be used as a learning tool in classroom. Element of increasing complexity was used by created three levels in the game that is easy, medium, and hard. These are different in terms of the number of animals and their walking speed. Figure 10 shows the screenshot of level menu page.
6. Testing and Result
Usability testing technique has been used to evaluate the prototype by testing it with 15 students. The purpose of this testing is to identify any usability problems, determine student’s satisfaction, and ensure that the prototype is in line with standard curriculum.
The usability testing was conducted using observation and interview method. During the user testing session, researcher and teachers conduct observation while students play the game. Meanwhile, at the same time they also filled out the checklist question regarding game interface.
These questions contain five quality components of usability [15]: learnability (Q1), efficiency (Q2), memorability (Q3), errors (Q4), and satisfaction (Q5) (see Table 3). Then semistructured interviews were conducted after the users testing session was done. The interview was conducted with the teachers to answer a set of question regarding content of the prototype.
The observation result shows that all students can perform the tasks given. Only one student did not pass one part of “error” section (Q4), that is, “menus do not mislead the users.” Table 3 shows the observation result.
During interview session, teachers provide the suggestion for improvement of the prototype including the error part that the student encountered during his work. Teachers suggest providing a text in any button on the game interface, so it would not mislead players in their action. Another suggestion is to add voice over to the instructor character on the game. In the term of game content, teachers agreed that the content of prototype is in line with their standard curriculum.
7. Conclusion
In commercial circles, we see games pushing the envelope as it relates to such technologies as Microsoft’s Kinect and Nintendo’s Wii. These games have pushed the boundary in terms of game play, digital storytelling, and graphics without consideration about the content, whether it is suitable for education or not. This study proposed a model for NUI game-based learning to help game developers in designing NUI game. With the existence of this model, hopefully it will support those technologies in processing to be integrated and used as learning tool in classroom. So that it can enhance the classroom interactions, increase the classroom participation, and create opportunities for interaction and discussion to achieve learning goal.
One limitation of this work is that we did not compare our model to other methodologies or frameworks. However, it is our intention to do so, as part of the future work, and we hope other researchers to continue the robust research about NUI educational games.
Competing Interests
The authors declare that they have no competing interests.
Acknowledgments
This research was funded by University Grant GUP-2015-004.