Abstract
This paper focuses on developing, testing, and examining the Proagents multimodal learning environment to support blind children's explorative learning in the area of astronomy. We utilize haptic, auditive, and visual interaction. Haptic and auditory feedbacks make the system accessible to blind children. The system is used as an exploration tool for children's spontaneous and question-driven explorations. High-level interaction and play are essential with environments for young children. Proactive agents support and guide children to deepen their explorations and discover the central concepts and relations in phenomena. It has been challenging to integrate together in a pedagogically relevant way the explorative learning approach, proactive agents' actions, haptic perception's possibilities, and the selected astronomical phenomena. Our tests have shown that children are very interested in using the system and the operations of the agents.
1. Introduction
Today’s children are immersed in a world of play. Much of their everyday activities center around different kinds of digital facilities designed to bring them joy through playing. The dominating role of entertainment affects their relation to computers as well as their learning. In this paper we emphasize exploratory learning, in addition to the playful elements, in order to support children’s conceptual learning and thinking. With this, we strive for a combination where entertainment is not so dominant, in line with the “hard fun” (instead of “soft fun”) concept of Papert [1]. To accomplish this, we utilize multimodal interaction with novel devices and proactive software agents to provide just-in-time assistance for both sighted and blind children.
The exploratory learning approach can be examined by studying the phenomenon that will be the subject of investigation, the exploration activities themselves, and the learning environment where human and computer support is emphasized. The main aim of exploratory learning is to make it possible for children to understand more deeply the phenomenon in question so that they can predict and explain the phenomenon and enrich their knowledge, ultimately achieving conceptual change if needed. This means that children’s exploratory activities should not be restricted only to hands-on activities, but rather aim at understanding the underlying theory and central abstract concepts as well as their relationships within the phenomenon. Attention should be directed to the way children explore the phenomenon and what kind of exploration strategies they use. Essential to a fruitful inquiry process is the ability to pose questions, construct explanations and hypotheses, and to search for new knowledge. The meaningfulness of the tasks and children’s interest are central starting points for such an inquiry process. However, for the development of exploration strategies and in order to grasp the central ideas behind the observed phenomena, children often need support for their explorations. Other children and adults play important roles in supporting knowledge construction. Materials, tools, and technological equipment can also help to structure the inquiry process.
To support children’s exploratory and conceptual learning of astronomical phenomena we constructed a multimodal learning environment that is based on dynamic haptic, auditive, and visual interaction. Haptic feedback provides the users with an additional presentation channel, making the system accessible for blind children, and increasing the level of engagement also for sighted children while they are studying the astronomical phenomena. The construction of the computer environment is based on the notion that children could use the system as an exploration tool for their spontaneous and question-driven explorations. Attention is paid to the support which the children might need in their explorations. Support in this computer environment is implemented through built-in proactive agents that aim to create situations that stimulate children’s active and constructive inquiry processes. Agents provide personalized support for children, try to anticipate the problems they might have, and suggest pedagogically relevant new approaches.
In the following, the starting points and theoretical principles of designing and constructing the multimodal learning environment will be examined in more detail. Of specific interest is the introduction of pedagogical bases from the perspective of exploratory and inquiry learning in which human and computer support plays an important role.
2. Astronomical Phenomena as Content
When selecting the natural phenomena for the proactive computer simulation system of the research in question it was essential that the phenomena were important and significant with regard to life. The simulated phenomena have to awaken sufficient interest in the children and efficiently utilize possibilities offered by the new computer technology. Some of the most suitable phenomena are those that can be easily and illustratively presented in no other way, such as phenomena linked with space, the atmosphere, and many other phenomena in elementary physics. There has to be a coherent theory of the phenomenon, which in turn has to include important, central, and abstract concepts. A final selection criterion for the chosen natural phenomenon can be seen as its conformity to scientific principle which should form a clear, well-organized knowledge structure and theory. These aspects present a strong and defined foundation for the computer simulation (see, e.g., [2]).
In this research, the phenomena chosen for computer simulation were our solar system, the interrelations between the Earth and the Sun, the Earth, the atmosphere, and the interior layers of the Earth. Although the phenomenon as a whole is abstract, it is possible to concretize it by using the computer-based simulation. The core properties and features of the phenomenon can be built into the computer-based simulation, which then will assume a concrete form in the simulation run. The selected phenomena have also been modeled so that visually impaired and blind children can easily investigate the phenomena using a haptic device.
Earlier studies on children’s conceptual learning in the area of astronomy have shown that children often face difficulties when they are learning these phenomena. This is because the early observations children make of their natural surroundings, and from which they construct their understanding, do not often correspond to the scientific view. The studies have shown that children’s initial conceptions about the physical world are very deep rooted and difficult to change by means of instruction (e.g., [3, 4]). In the area of astronomy, children’s naïve, everyday conceptions have been widely investigated and are already well documented. Children’s knowledge about the Earth, especially, with regard to the Earth’s shape, is one of the most studied phenomena in this area (e.g., [3, 5–7]). For example, Vosniadou and Brewer (e.g., [7]) have found that children may start with an initial model of the Earth, in which the earth is a flat object. This model is supposed to be based on children’s everyday observations of their surroundings. Later, children may form “synthetic models.” These models are formed when children try to combine the scientific knowledge, received, for example, from instruction, into their intuitive models. The synthetic models Vosniadou has found include the flattened sphere model (people live on flat parts of a spherical earth), the hollow sphere model (people live inside the spherical earth), and the dual earth model (a flat earth where we live and a spherical one in the sky). According to Vosniadou, to adopt the scientific view of the spherical Earth, children often need to revise their existing knowledge structure and to change some of the entrenched assumptions or beliefs in which the concept of Earth is embedded. The process of restructuring is slow and gradual, and since children try to retain as many of their assumptions as possible or to change them only partially, misconceptions are very likely [8, 9]. Ikospentaki and Vosniadou [10] have included blind children in their studies, and found that blind children also have initial or alternative models of the Earth although they lack empirical, optical data from their surroundings.
Earlier research on children’s conceptual thinking has included phenomena such as the alternation of day and night, seasons, and phases of the moon (e.g., [5, 11, 12]). In all of these phenomena, a number of intermediated notions between the initial impressions and scientific notions have been discovered (e.g., [5]). Although the more naïve views showed reduction as the age increased, misconceptions still seemed to persist in many students up to 16 years of age.
In sum, these studies show that children may have various notions with regard to the shape of Earth and other astronomical phenomena, and that these notions are very persistent and difficult to change. They are also connected to one another. For example, Vosniadou et al. [9] have stated that the spherical shape of the Earth is not possible to understand without knowledge about gravity. Furthermore, to understand the scientific explanation of the day/night cycle, children must first understand that the Earth is a sphere. The spherical shape of the Earth and gravity are also prerequisites for understanding the spherical arrangements of layers inside the earth [9]. These findings have supported and provided background for the selecting of the microworlds for the simulation.
3. Explorative Learning and Play
Contemporary findings from cognitive science studies have demonstrated the importance of the individual’s or learner’s own activity and interest in learning. For example, based on the knowledge and results concerning the plasticity and self-organizing of the human brain, it can be said that the more complicated the phenomenon or matter in question, the more the learner’s own active explorative action, like analyzing and organizing the material, is needed for understanding and comprehending. Up to certain limit, when the optimal level of attention and activation prevails, the self-organizing and shaping of brain takes place and models are constructed in the brain. Through perceptions, experiences, and active mental work, the memory traces corresponding to these models grow and strengthen, and knowledge structures are formed (see, e.g., [13–15]). Furthermore, when considering the acquisition of expert skills in some area, the organization of domain-specific knowledge structures and the learner’s own active involvement and interest become even more important.
Playing has a significant role in young children’s learning. Children are often naturally interested in various phenomena around them and ask questions and explore their surroundings. Many scientific concepts, knowledge, and skills develop throughout the early years through the child’s active exploration and play. The explorations start with very concrete and small things, for example, when a small child drops things out of his/her pram. Soon this turns into a game where, after dropping, the child waits an adult to pick the toy up. Even very young children know where to look for the toy after dropping it; they look down. Through their experience they have learned a theory about the world around them; that is, when you release your hold of a thing it falls (on the ground) [16]. Later on, the children’s explorations can take an even broader form as they take place in social interaction with other children and include ever more complicated matters. Besides constructing an understanding of the surrounding world, through their early explorations children also develop many scientific skills; they identify similarities and differences, and group things together. Early explorations are often seemingly unsystematic and unproductive, but as a child matures, exploratory skills also develop which then enables more skilled exploration and investigation to occur. The exploratory skills such as observation, raising questions, classification, and hypothesizing are important first steps in the development of other skills in the scientific process, especially planning, predicting, and investigating [16].
In science education, the role of children’s interests and questions have become current themes through constructivist and socio-constructivist inquiry and project-based approaches that aim at fostering research-like processes of inquiry in education (see, e.g., [17–19]). In these approaches, learners are encouraged to initiate their own questions and hypotheses. These are then investigated under the teacher’s guidance. The aims of inquiry learning at schools are related to learning more deeply about the phenomenon in question, namely, that the inquiry would concentrate on the key concepts in the phenomenon and understanding the theory behind the phenomenon. Through inquiry children can also learn the research skills, develop research strategies and metacognitive skills [20]. Often the inquiry process is described as a process that consists of posing problems, constructing explanations and hypotheses, searching for new information, and generating new questions as well as explanations. The progressively deepening nature of the inquiry process has been emphasized as well as the collaborative mode of learning. Moreover, information and communication technology has been used to facilitate the inquiry processes [21–24].
Inquiry or exploration activities that aim at understanding more deeply the phenomenon in question are deliberate processes, and in many cases some cognitive and socio-cultural support is a necessity. In schools, teachers have an important role to motivate and support students in their inquiry. In exploration-based science instruction, it has been emphasized that more effort is placed on students’ thinking processes rather than on the need for correct answers, and that enough time must be given to the exploration of key concepts in one subject matter area [9]. It has also been considered important that students are provided with opportunities to work with phenomena instead of only watching teacher demonstrations, and that some cognitive scaffolding is available to help the students to find new and alternative ideas (e.g., [25, 26]). It has also been considered essential to pay attention to the order of acquisition of concepts in a given subject matter area. Teachers can take this into account when scaffolding the explorations. Meaningful and theoretically relevant experiences as well as providing models and external representations are important in clarifying the scientific explanations [9]. Models and external representations offer students opportunities to explore aspects of phenomena in other than linguistic form and they contribute for example the comprehension of complicated phenomena through providing visual presentations of interrelations in phenomena.
3.1. Computer Supported Exploration
Considering computer programs from the standpoint of the amount of support a child needs in exploration, we think that tool programs, as well as simulation programs, may leave the learner alone with his/her problems and the exploration process can be stuck for a long time. Of course, if a teacher or an adult is available, she/he can come to rescue, but often the teacher also has other learners to attend to. One possibility for the learner is to ask for help from more advanced peers in a classroom or in a networked environment. In fact, the web can be seen as an especially empowering environment for inquiry learning [21].
Knowledge-based programs (formerly called intelligent tutoring systems) are an interesting solution to the above-mentioned problem of support. These programs contain and maintain knowledge on the teaching domain, pedagogical approaches, as well as the individual learners, and present their support for the learner by combining these matters into guidance to assist the learner when she/he gets stuck or asks for help. The communication capabilities of modern tutoring systems are rather advanced. For example, some of these systems hold a conversation in natural language and in this way help students construct answers to deep-reasoning questions (e.g., AutoTutor system, [27]). AutoTutor is not merely an information delivery system, but it coaches students in actively constructing knowledge [27]. Some other tutoring systems, such as the Cognitive Tutors for mathematics from the Carnegie Learning Company, which is already in classroom use, excel in providing versatile interaction opportunities (in addition to diagnostic interventions) in problem-solving situations [28].
Lately, these tutoring systems are orchestrated from several coworking program entities, called intelligent agents. Agents can function “behind the scenes” as an implementation technique, or manifest themselves to the learner as animated characters on the screen. Much attention has been paid to animated pedagogical agents or characters presenting the support of the tutoring systems. Indeed, it is true that a vivid figure on the screen can make a difference to children, also in learning (see, e.g., [29]). We have constructed an agent-based learning environment [30] with several teacher agents and learning companion agents for the learner to select. Having the possibility to change your teacher or learning companion during the sessions helped the learners to find coworkers that suited their preferences [30]. However, we tend to agree with Graesser et al. [27] that it is the “message (the dialogue moves of the tutoring system)” that is more important than the “medium (the way it is presented to the learner).” This emphasizes the role of the “just in time” and informative dialogue moves, which are also in focus in proactive support for explorative learning. In addition, the intended users of our prototype are restricted to only a few of the typical input channels of computer programs, which set further challenges for the agent environment.
Visually impaired or blind children pose a tough problem for computer assistance. The program cannot rely on graphics, but the interface must depend on sound or the sense of touch (haptics). The degree of the disability, of course, dictates the environment design. However, it is clear that children with severe visual impairment also want to have the opportunity to access the learning environment, such as a game, alone and choose by themselves [31]. A design might include special tactile sheets for input and sound as feedback from the system, as in the case of Buaud et al. [31]. Baloian et al. [32] discuss general principles for educational software for visually disabled, and suggest an (intelligent) tutoring component in their architectural model. Their example system uses both the computer and LEGO blocks, both sound and touch.
3.2. The PICCO System as an Example of a System to Support Exploration in Astronomy
Kangassalo (e.g., [24, 33]) has examined the knowledge construction and conceptual understanding of phenomena in astronomy by pre and primary school children in learning environments where children have had possibilities to explore the astronomical phenomena using a computer-based multimedia simulation program PICCO [2, 34, 35].
The natural phenomena chosen for the PICCO program consist of the mutual relations of a number of phenomena, as well as the temporal and spatial relations of objects. The phenomenon chosen for the program is the changes in the sun’s light and warmth as experienced on Earth in relation to the mutual relations of the Sun and the Earth on the spatial level. The key concepts and phenomena in the program are time, night and day, the seasons, the Earth and the Sun, and space. The contents of the program are structured in such a way that the phenomenon can be first studied on the surface of the Earth, and children may proceed along various paths into the different areas of the program, according to their own interests. The areas include the mutual relations of the Earth and the Sun in space, the solar system and the planets, the mutual size and distance relations of the Earth and the Sun, and our solar system as part of the galaxy. The children may explore, for example, on the earth level what happens in the phenomenon in the environment at different time, in different months, and at different compass points. The phenomenon can be studied with the help of pictorial symbols of time and space, as well as with special research tools in the program, a dictionary, and a space shuttle. When studying the phenomenon from the surface of the Earth, for example, the seeming trajectory of the Sun and its position in the sky can be studied with the clock and the calendar, the changing of light and dark on different days of the year, the changes taking place in nature over months and seasons. At any moment a child can proceed up to the space level and explore the interrelationships of the Earth and the Sun at the space level. The program is a pictorial entity that proceeds in the form of events that are described realistically, as seen from the surface of the Earth, and on the spatial level based on analogy models. Reading or writing skills are not required for using the program. The child may proceed with studying the phenomenon according to his/her own interests and thoughts either alone, in pairs, or in small groups [2, 34, 35].
On the basis of the research experiments in both preschool and primary school teaching, it can be concluded that PICCO worked well as a tool for the children’s spontaneous and independent exploration. Exploring the phenomenon with PICCO supported the construction of the children’s knowledge of the phenomenon and directed learning towards established scientific knowledge. The children’s perceptions of the phenomenon as an entity and their outlinings of the mutual relations of the concepts and the link-ups developed and became more structured. As a synergy of PICCO and traditional teaching, considerable knowledge construction was detected. The children used a variety of research strategies while studying the phenomenon with the simulation program. The children’s conceptual models on the phenomenon acted as a basis through which they proceeded with exploring the phenomenon with PICCO. The research strategies children applied in their exploration were more advanced the more developed the children’s conceptual models of the phenomenon were (e.g., [24, 33, 36]).
4. Proagents Multimodal Learning Environment
Based on the theoretical and pedagogical background and research results with PICCO, a new Proagents system was designed. The leading design principles, in addition to the pedagogical principles, were the utilization of multimodal interaction (visual, auditory, and haptic) and proactive agents. The Proagents multimodal learning environment supports pre and primary school aged (6 to 8 years old) children’s exploratory and conceptual learning in the domain of astronomy. Of special interest to us has been the construction of support for children’s explorations and conceptual learning when they are using the system.
Multimodal interfaces [37] make use of the different senses and actions of the user. Multimodal systems have only recently been introduced in applications that are directly usable by a common computer user. Research on multimodal interfaces involves research on both human senses and capabilities, as well as how to construct such multimodal systems. We focused on the interaction design and content of the system, the highest level of which is visible for the user as the concrete multimodal user interface.
Disabled users may really benefit from multimodal interaction. There is a great amount of different disabilities and impairments, but in general adding new modalities, such as making use of a wider variety of senses, makes it possible for many users with special needs to make use of modern technology. This is why we have selected visually impaired children as the target users of our multimodal learning environment.
4.1. Pedagogical Approach of the System
The pedagogical approach of the system is based on exploratory learning. This means that a child can explore the selected astronomical phenomena independently, according to his/her own interests and questions. There are no rules or pathways on how to proceed in the program. The environment consists of six microworlds that a child can explore, see Figure 1. The user starts from the central station. From there she/he can move to one of the six microworlds. When the user is navigating from one microworld to another, she/he must travel through the central station. This lessens the likelihood of getting lost in the environment because the user is always only one step away from the central station. The microworlds are the solar system, the Earth and the Sun, the Earth, the atmosphere, and the interior layers of the Earth. The selected phenomena include central and abstract concepts related to the Earth’s shape, to time and space, to the seasons, and to the alternation of day and night. The concepts and phenomena in microworlds are many ways connected to each other, and together they form a coherent theory.
The theoretical foundation and pedagogical basis for children’s explorations are derived from the inquiry models that emphasize the role of questions as a starting point for the inquiry. One such model is the interrogative model of inquiry. This model was originally developed for the purposes of the philosophy of science (see [38–41]), but it has also been used to represent knowledge seeking in educational contexts (e.g., [18, 21]). In the interrogative model, scientific procedure is viewed as information seeking by questioning. More specifically, inquiry is defined as a series of questions the inquirer poses during his/her inquiry process, either to nature or to some other source of information. The inquirer tries to derive an answer to his/her initial question or problem by using his/her existing knowledge and by formulating and seeking answers to smaller questions. The acquisition of new knowledge raises new questions that have to be examined. By choosing the questions, the inquirer can direct the course of the inquiry according to his/her own plans [21, 38]. According to the interrogative model, an inquiry can be conceived as a dynamic, question-driven process of understanding [18].
In this research, applying the interrogative model means that a child’s learning is viewed as an active process guided by his/her own questions and previous knowledge. The selection of the approach is largely based on Kangassalo’s earlier research results [24, 33] with the PICCO computer environment, where a child has been seen to progress in his/her exploration process step by step on the basis of his/her earlier knowledge base concerning the phenomenon in question. In this project, we aim to continue the PICCO project by constructing support for children’s explorations. The support aims at encouraging the formation of questions in child’s mind as well as the process of seeking answers and explanations.
4.2. Proactive Agents as a Scaffolding Tool
It is possible to implement the scaffolding using software agents, independent program entities that can provide knowledgeable “just-in-time” support for learning. There are several existing agent applications that support the inquiry process and conceptual change in learners studying science. For example, Ting and Chong [42] have constructed an animated pedagogical agent that acts as a catalyst coaching learners through the scientific inquiry. White et al. have reported several studies on multiagent environments fostering scientific inquiry, conceptual change, and reflective learning; see for example, the Inquiry Island environment [43].
To support children’s explorations in the system, we have utilized proactive agents as a scaffolding tool. Our program utilizes many kinds of agents. In this article, we describe the pedagogical agents that communicate with children. Each microworld in the program has its own assisting pedagogical agent. The agents are different imaginary characters, and they all have different names and voices. Since the system is developed also for visually impaired and blind children, the agents’ operations are based mainly on auditory feedback.
The starting point for constructing the agents’ support has been that the agents would support children’s explorations in the program starting from the child’s own activity, interest, and questions. For this reason the agents support the explorations mainly by making questions and suggestions to the child, and the child him/herself can choose whether to take the agents suggestions or to continue with his/her own exploration. With their suggestions and questions the agents aim at deepening and extending the child’s explorations in the program, and to awaken questions in a child’s mind. Decisions regarding the appropriate times for suggestions are based on, for example, the child’s exploration time in a microworld and the child’s exploration path in the program.
The agent’s suggestions assist the child to find the central phenomena and concepts that are connected to the microworld the child is currently exploring. They also guide the explorations from one microworld to another, and in this way support the discovery of relations and explanations in the selected phenomena. From the viewpoint of conceptual learning, the agents guide the explorations from familiar everyday observations towards the causes and scientific explanations of phenomena. For instance, if the child is exploring the solar system and has already examined the different planets and their properties, an agent might suggest to the child that she/he could find the planet Earth. After that the agent may challenge the child to consider why only the Earth has people and animals on it, and suggest that the child might like to explore the Earth even closer. Furthermore, when exploring the Earth an agent may direct the child’s attention to Earth’s gravity with a comment like “did you notice when you traveled with your space shuttle you were pulled to the Earth’s surface?” It can also challenge the child’s thinking with arguments like “what happens to different objects when you throw them into air or drop them?” and offer explanations on gravity.
In summary, the agents try to guide the children to elaborate their previous knowledge through their questions and suggestions, and encourage them to examine the properties and relations in phenomena. The proactive agents also aim to help the children become conscious of their own exploration and thinking. The agents allow children to explore and proceed in different ways, and the child him/herself can continuously choose either to listen to what an agent wants to say or to ignore him.
4.3. Multimodal Interaction
The learning environment is based on using a Reachin Display System [44] which integrates a PHANTOM force feedback device from SensAble Technologies [45] with a stereo monitor and supporting systems (see Figure 2). The PHANTOM device has a thick movable stick which provides haptic feedback. By using the stick, a child can feel three-dimensional entities. For visually impaired children, haptic and auditory modalities are the main presentation channels, and for children with normal eyesight, the system provides opportunities to receive and use the information obtained through multiple senses.
Haptics often involve active touch (intentional actions) that a person wants and chooses to do. In other words, when using the system the learner can control his or her actions and the speed of exploration. This is in contrast with passive learning, such as watching a video or a teacher demonstration, where the learner does not have to make any decisions or initiate actions. When comparing haptics, for example, to vision in the perception of objects, vision is usually considered superior although there are some important exceptions. Visual perceptions allow the learner to take in a lot of information at one time and it is more rapid and holistic than haptic perception. Haptics, on the other hand, help learners detect the properties of texture and some microspatial properties, like elasticity, compliance, and pattern [46]. In our system, it has been possible to represent the concept of gravity through haptics. Furthermore, features of the Earth’s surface, such as oceans and the ground, have been possible to create as one object of exploration.
The proactive and adaptive functionality of the system is implemented by carefully modeling the astronomical phenomena as well as maintaining student profiles. Based on this long-term logging of user actions, rule-based agents initiate proactive actions. The inference and learning capabilities of the agents will enable them to produce proactive behavior, such as to suggest autonomously interesting or important information at appropriate times. Since the targeted users of the system have either just started school or are in their first years of school, the most useful way to provide an agent’s feedback is speech. This is also directly suitable for visually impaired children.
Haptic modality is also involved in agents’ operations, for example, when an agent wishes to say something, the stylus in the child’s hand shakes. The child can then decide whether he or she wants to hear the agent’s message by pressing either a “Yes” or “No” button. In addition, agents may also refer in their messages to the sense of touch when the child is examining and feeling a microworld with the stylus.
Figure 3 shows the low-level technical structure of the agent-based system [47]. The controller is built on top of the Reachin API [44] which provides a means to interact with the 3D simulation environment and the haptic devices, and our own simple sound system. The basic agent architecture can be divided into three separate functional components. The system can be seen as the realization of a typical message dispatcher architecture where MessageChannel is the central dispatcher providing a centralized means for passing messages between agents around the network. The agent containers (and thus agents) are connected to the agent system via MessageChannel. The third component is the actual application that is handled by the controller. The functionality and structure of MessageChannel, agent container, and agents is loosely based on FIPA agent specification (http://www.fipa.org/).
The agent-based system architecture is used to construct higher-level agents that handle multimodal input and output [47]. The database agent stores application-dependent information and notifies the agents that have subscribed to be informed of a certain change in the database. The rule engine agent has been built on top of the CLIPS rule engine [48] and manages rule-based events in the user interface. Filter agents can be used to process originally low-level input in higher levels following the basic pipes and filters functionality. The controller agent handles the central control of the application and manages the logging agent that stores events in the system.
A detailed technical description of the system can be found in our earlier publications [47, 49]. The present paper adds to these publications especially in the areas of pedagogical design and testing of learning, usability, and interaction in the environment.
4.4. The Microworlds
In the following, the program’s microworlds are described shortly. As mentioned in the previous sections, the system consists of six microworlds that the child can explore according to his/her own interests. In the solar system microworld the child can examine the planets in our solar system. The solar system is depicted as a whole on the screen; the planets, their orbits, and the sun. As the child touches one of the orbits, the program tells the child which of the planet’s orbit is it. The orbit can be felt under the stylus as a groove, and a blind child can move the stylus along the groove. This design decision was made based on the need to provide some guidance for a blind child navigating in a three-dimensional space. We experimented with different alternative methods. In our pilot tests, the selected 2.5-dimensional representation helped the children to use the system and still get the benefit from three-dimensional haptic forms and textures. When the child touches the planet with the stylus, the program tells him/her what planet it is and where it is located in relation to the Sun. It is also possible to listen to more information on each of the planets by pressing the planet with the stylus.
In the Earth and the Sun microworld, the child can study the Earth’s revolving around the Sun, as well as the changing of the seasons in Finland at the different stages of its revolution. As visual feedback, the screen shows the Sun, the Earth, and the Earth’s orbit. When the child touches the various objects in the microworld, the program tells him/her where she/he is located. The orbit is presented as a groove, which enables the blind child to follow the orbit. The program also informs the child, when she/he finds the Earth, and the child can then move the Earth on its orbit around the Sun. At the different stages of the orbit, the program plays the sounds typical to the season in Finland. The program also tells about the Sun’s light and warmth in each season.
In the Earth microworld, the child can study the round shape of the Earth as well as the Earth’s surface. The Earth can be felt with the stylus as a three-dimensional round object. When touching the surface of the Earth, it is possible to feel the differences between solid ground and the oceans. Haptic feedback is supported by auditive feedback; when touching the surface of the Earth, one can hear the sounds of human habitation and nature; at the sea, one can hear sounds typical of the ocean (waves, seagulls). The Earth also has gravity that can be felt with the stylus as a light pull towards the Earth. Gravity is one of the things that were possible to represent very well through haptics. The user can also rotate the Earth around its axis by moving the stylus to the right of the Earth.
In the Bowels of the Earth microworld, the child can explore the insides of the Earth. The various layers of the Earth are represented as a cross-section of the northern hemisphere. The layers can be explored by touching them with the stylus.
In the atmosphere microworld, the child may study the Earth’s atmosphere from the surface of the Earth to the upper layers of the atmosphere. Exploring the microworld is first and foremost based on auditive feedback. The haptic feedback is almost unnoticeable and light, and it aims to create a tangible “feeling of air.” The child can freely move with the stylus through the different layers of the atmosphere. The program also tells the child, when the child him/herself so desires, about the characteristics and the importance of the atmosphere.
The study room is a room with six doors. When the child presses one of the doors with the stylus, the program gives the child a statement related to gravity, and the child can answer “yes” or “no” by pressing a button. There are grooves that lead to the doors, on the floor of the research laboratory; and they assist the blind child in finding his/her way to the doors with the help of haptic feedback.
5. User Studies: Testing and Evaluating the System with Children
Both blind and sighted children have participated in the planning and testing process of our system from the very beginning of the research project. The development and testing of the system continued throughout the project. There were four kinds of tests included in the system development: the testing of the system’s manuscript, tests at the usability laboratory, children as expert evaluators, and a study of children’s exploratory learning. In this section, these tests are presented as well as some results of the tests. The results and experiences we received from the pedagogical agents supporting children’s explorations are presented in the next section.
5.1. Testing the Manuscript of the System
In the first phase, we tested the manuscript for the agents’ operations with children. The testing included interviews of ten 5- to 9-year-old sighted children. The interviews were conducted mainly at a day care center, and the children were interviewed individually.
The interviews proceeded as follows. First, the children were asked about their interests and questions with regard to the selected phenomena. After that the interview concentrated on one or two microworlds. The child was presented with general idea(s) about the micro world(s) and some short narrations of the agents’ operations were read aloud. Limiting the discussion only to one or two microworlds was considered necessary so that the interviews would not be too long for the child. However, since there were three interviewers, all of the microworlds were evaluated by children. When presenting and discussing the manuscript and the agents’ operations, some assistive materials were used (such as a picture of the solar system, a globe, etc.). This material helped to concretize the phenomena and the manuscript. The child was then asked how he/she liked the manuscript, how the narration could be improved, and whether he/she thought that something could be added or removed from the manuscript. Some separate utterances and sentences that were designed for the agents were also read aloud, and children were asked to evaluate if there were any difficult words in these utterances.
One interview lasted about thirty minutes, and the interviews were recorded on tapes. This testing at the very beginning of our developing work helped us improve, for example, the lines we had designed for the agents, and at the same time, it gave us insight into children’s interests and knowledge with regard to the phenomena chosen for the computer program.
5.2. Testing at the Usability Laboratory
We have also tested the usability of the system with children. These tests have been carried out in the usability laboratory and at the school for visually impaired children. The tests in the laboratory have been organized whenever some new solutions have needed testing, and both visually impaired and sighted children have participated in these tests at different phases of the system’s development.
Two researchers participated in the testing situations. One of them conducted the actual test with the child and the other was responsible for observing and videotaping the situation. The researcher who conducted the testing situation had a kindergarten teacher’s training and experience in working with children.
The children came to the laboratory with their parents. After the laboratory rooms were introduced to the child, the researcher talked with the child and the parent. The purpose of the discussion was to inform the child and the parent about the goal of the test, to tell about the system, and to have the child feel secure and comfortable.
The test began by familiarizing the child with the PHANTOM device. After that, the child was given a tangible model of the program’s navigation tool. The navigation tool is called “Central Station,” and it’s a place where a child can choose which microworld he/she would like to explore. The researcher guided the child to explore the tangible model first with hands and then with a plastic stick. The plastic stick was modeled after the PHANTOM’s stylus. At the same time the researcher explained to the child the function of the navigation tool (“Central Station”), and guided him or her to examine its shape. Familiarizing the child with the navigation tool using the tangible model helped the children comprehend the general structure of the program. This was important, as we wanted to support children’s independent exploration.
After that, the child could start using the program. The goal was that the child would use the program as independently as possible. However, guidance and help was offered whenever the child needed it. To some extent, the researcher also tried to elicit children’s ideas and comments during the use. The researcher, for example, sometimes asked the child to describe what he or she felt with the stylus, and made suggestions that encouraged the child to explore and examine the different objects in the microworlds.
After the child had used the system, he/she was briefly interviewed. The aim of this small interview was to hear the child’s comments about the program: how easy or difficult it was to use, what the child had liked and disliked in the program, what could still be added to program, and so forth.
All the tests were recorded on video and afterwards the usability of the tested microworlds was evaluated. The evaluation of the usability in these tests meant that we mainly observed what kind of solutions supported children’s independent exploration and in what kind of situations children most often needed researcher’s help.
5.3. Children as “expert Evaluators”
Two tests were carried out at the school for visually impaired children. The first of these tests included seven 12-year-old visually impaired children. The microworlds that were tested were the solar system, the Earth, and the study room. The aim of this testing was to use the system in natural surroundings and to receive as much feedback as possible from the usability of the system. The children in this test were older than the actual target group of our system, and the children’s task in this test was to act as “child experts” to give comments and feedback about the system and to evaluate how they thought that smaller children than they would be able to use the system. In all test situations the children used the system one by one, and the researcher sat next to the child and assisted him/her when necessary.
Before the actual tests, the researchers introduced themselves to the children in their classroom, and told them about the test. The purpose of the test was emphasized: we wished the children to use the program, and to tell us what they thought about it, what they thought about the selected astronomical phenomena, and how they thought that the program could be improved. The children were also told that the program is designed for younger children than they were. The important thing was to create and maintain the children’s motivation for the evaluation task, and to describe the device and the testing situation beforehand so that the children would feel secure and confident about the test.
Similar to the laboratory tests, two researchers participated in the testing situation. The children’s use of the program was videotaped. The test took place in an empty room near the children’s classroom. Each child was reserved a lesson period of 45 minutes for the test, but the time children used the system varied. This is because the use of the system was based on the children’s own interest. At first, the child was familiarized with the system. Then the structure of the program was explained to the child with the help of the tangible model of the navigation tool (Central Station). After that, the child was allowed to explore the program freely. The researcher sat next to the child, and provided help whenever the child needed it. The researcher also tried to elicit the children’s comments and thoughts during the use of the system, and asked the children such questions as “if you’d like to find some particular planet, how difficult would it be?” and “how did you understand what the agent just said?” The children were also briefly interviewed directly after the use. Below are some examples of the comments children gave regarding the agents and their operations after they had used the system.
“You could say, for example, how cold it is on Pluto…if possible?” (solar system microworld)
“Maybe more information about the Sun and its structure—a small child may think that the Sun is solid, although it indeed is not.” (solar system microworld)
“Earth Giant (the agent) could tell something about the people who live on the continents you are currently exploring.” (The Earth microworld)
These were just some examples of children’s comments. In sum, children were very motivated in evaluating the system, and their comments and suggestions seemed well thought and also realistic with regard to realizing them. The children’s comments regarding the overall usability of the system were also very valuable to us, especially, when we were thinking that also smaller children should be able to use the system.“I would add there also more challenging questions, really difficult ones, so that there would be questions of various levels of difficulty.” (Research laboratory)
5.4. Study of Children’s Exploratory Learning
The final test was carried out at the school for visually impaired children where two 7-8-year old blind children participated. Afterwards, a third child from this same age group participated, and this test was carried out in the usability laboratory a few months later. In this test we had three microworlds available: the Earth, the solar system, and the Earth and the Sun. One of the aims of this test was to examine blind children’s exploratory and conceptual learning when using the system.
The two children who participated in the test at the school for visually impaired children used the system twice, for about a single lesson period of time (45 minutes) on successive days. The third child used the system at the university’s usability laboratory once also for approximately 45 minutes. At first, a child was familiarized with the PHANTOM device, and a tangible model of the central station was also presented (see the description of the previous tests). After that, the child could use the program and explore the phenomena according to his/her own interest. In this test, we also had plastic tangible models of all the micro worlds. As the child explored a microworld, she/he was able to get a general picture of what he or she was exploring by touching the plastic model. As in the previous tests, the researcher helped the child whenever he/she needed help in using the system. In addition, the researcher encouraged and supported the child’s explorations by posing questions and sometimes directing the child’s attention to central objects or events in phenomena. However, the use of the program was always based on child’s own interest and independent exploration.
The research data gathered included video recordings and log files of the children’s use of the program. The log files provided information about the child’s exploration pathways. The child’s comments, questions, and other expressions as well as the researcher’s guidance could be observed from the videotapes. To get an accurate picture of the child’s exploration process, these video recordings were also transcribed next to the log files.
6. Results and Experiences on Pedagogical Agents Supporting Children’s Explorations
In this section, the results and experiences we have had of the pedagogical agents that were designed to support children’s exploration are discussed from the basis of the tests carried out. The discussion concentrates on the experiences we received from the final test carried out with three visually impaired 7-8-year-old children. The experiences and results are preliminary in nature because there were only three children who participated in the test, and also because during this test the environment was still under construction and the agents’ action did not yet include all the messages that were designed to them.
To analyze the experiences we got from the pedagogical agents supporting children’s exploration we transcribed the researcher’s and a child’s interaction from the video recordings next to the log files the computer had gathered. In this way it was possible to reliably identify the child’s comments, questions, and other expressions and actions with regard to agents’ messages from the data.
6.1. Children’s Responses to Agents’ Messages
During the children’s exploration there came several messages from the agents (child 1: 8 messages at the first exploration time, 12 messages at the second exploration time; child 2: no log files saved from the first exploration time, 10 messages at the second exploration time; child 3: 9 messages). The children chose to receive almost all the messages (92% of all the messages were received). Most of the agents’ messages were such that the agent told more information for the child about the object or phenomenon he/she was currently exploring (72% of all the messages the children chose to receive).
The children’s reactions and responses to the agents’ messages showed that the children were able to include the agents naturally in their exploration of the phenomenon, although the amount the child commented on or discussed the messages aloud varied. When examining the exploration process of child 1, who was the most talkative of the three children, from the point of view of his reactions and responses to agents and their messages, it seemed that the child became more interested in the agents and their messages during the exploration, and was able to include them better as a part of his inquiry as the exploration proceeded. During the first exploration time, this child received all of the agents’ messages that became available and commented on one of them aloud. During the second exploration time, the child answered one of the agents’ questions aloud, and afterwards, after receiving another message, he started to re-evaluate his earlier answer and thinking.
(in the Earth–the Sun Microworld:)
09:43:18Sunny Anneli (an agent):You
moved the Earth around the Sun quite fast with the stylus. How long do you
think the real planet Earth takes to travel around the Sun? You can think about
this while travelling around the Sun once again and I’ll tell you what it is
like here in Finland
during the different parts of your journey. Child1: and it took the two months. Researcher: mm. Child1: I think. Researcher: well we can circle the orbit again and as we circle we can listen.
(in the Earth Micro World:)
09:52:41Earth Giant
(an agent):You are now
exploring the earth, the planet in which we live. The earth’s circumference is
approximately 40,000 kilometers. This means that if you could
travel around the world by car, you would have to sit still for three weeks in a row. Child1:It takes less time than the round of the sun09:53:03Surface of earth09:53:03Ocean Researcher:mm.09:53:04Ocean Child1: It…. I think I guessed it a bit wrong.09:53:09Ocean09:53:09Ocean Researcher:well, what do you think about it now?09:53:09Surface of earth09:53:10Surface of earth Child1:Now I think if you took an airplane it’d take only two weeks.09:53:11Surface of earth.
There was also a phase where the child tested the agents’ function through pressing the “Yes” and “No” buttons to different messages and looking what happens. Towards the end of the exploration the child seemed to learn to expect to hear more information about the phenomenon from the agents. For example, when there was a message coming from the agents, he told the researcher that “I press “yes” so that I can hear information.” He also expressed a wish to hear an agent when he was in solar system and examined the Sun (“If a message comes hear, I will press “yes” ”). The last message from the agents elicited a question from a child, which could have lead to further exploration.
The two other children did not make many comments aloud. However, they listened to the agents’ messages very carefully, and the one child (child 2) commented and wondered at the low voice of the Earth Giant many times. He was also very interested in the narrative elements of the system as he made many questions and comments regarding, for example, vehicles at the research station. He also noticed when there was a long break in the agents’ messages (“Now the program has not said anything for a long time…. This must be such an object that here it needs so much…”).
6.2. Future Challenges
When considering the phenomenon as a whole, how it was modeled in the program and how the agents managed to support the children’s explorations from this perspective, it seems that the Earth and the Sun microworld (where the child can explore the Earth’s revolution around the Sun) was the most challenging microworld to model for the blind users as well as to support their explorations there. In this microworld the movement of Earth in relation to the static Sun, and the connection of this to the changing of seasons, were the main objects of exploration. Making the movement possible for blind children to explore was very challenging. On the basis of this small test it seems that at least children whose conceptual model of the changing of seasons is at a very initial level and does not yet include features of the scientific model would need more support for this microworld to be outlined, or the support needs to be constructed differently in order to make the perspective from which the relations of the Earth and the Sun are examined clearer.
We consider the experiences we received from the pedagogical agents supporting the children’s inquiry to be very promising, although there’s still some development work and more extensive tests to be done. The technical solution for receiving and rejecting the agents’ messages through pressing the buttons after the stylus in the child’s hand had shaken seemed to work well because the children learned this very quickly. At the time of this research experiment, the agents’ role as “information givers” became emphasized because of the nature of their messages at this developing phase. However, the children seemed very motivated towards the information given by the agents, and sometimes waited for the agents to tell them something. In future development work, it would be good to increase the amount of suggestions and questions in the agents’ operations, and to carry out more extensive tests. In this research experiment, the researcher acted as the children’s conversational partner and assisted the children in the use of the system. In the future it would also be interesting to examine how, for example, children discuss and explore the phenomenon together when using the system.
7. Conclusions
Information technology can be of great help in the understanding of astronomical phenomena. According to the earlier research it has also been found that children are very interested in these phenomena and they explore phenomena on the basis of their earlier knowledge. Children’s conceptual understandings have seemed to develop in the direction of the currently accepted scientific knowledge when children have had possibilities to explore these phenomena using computer simulation system PICCO program [34, 35]. These earlier research results concern sighted children [24, 33].
We have constructed a system that supports blind and visually impaired children. For them, information technology provides greater opportunities to explore abstract phenomena and their spatial relationships. We used haptic devices for producing haptic sensations, as haptic perception is an important exploration means for blind and visually impaired children. Proactive agents were used to support children’s explorations and conceptual learning in the domain of astronomy. It was challenging to construct and integrate in a pedagogically relevant way the explorative learning approach, proactive agents’ action, haptic perception, and the selected astronomical phenomena for seeing, visually impaired and blind children’s learning and exploration. Based on our results, the system is expected to support children’s explorative action, the formation of questions, the construction of knowledge, and the enhancement of interest in the selected astronomical phenomena.
Acknowledgments
We express our gratitude to the members of our research group and the children and their parents for participating in our pilot experiments. Special thanks to Janne Järvi, Rami Saarinen, and Jouni Salo for their valuable work in the implementation of the system. This research (Grants no. 202179 and no. 202180) was funded by the Academy of Finland as part of its research program on Proactive Computing (PROACT) for the period 2002–2005 (http://www.uta.fi/proact/).