Abstract
In recent years, virtual reality and augmented reality have emerged as areas of extreme interest as unique methods for visualising and interacting with digital museum artefacts in a different context, for example, as a virtual museum or exhibition, particularly over the Internet. Modern cultural heritage exhibitions have evolved from static to dynamic exhibitions and challenging explorations. This paper presents two different applications developed for the Herbert Museum and Art Gallery that make the user’s experience more immersive, engaging, and interactive. The first application utilizes mobile phone devices in order to enrich the visitors experience in the museum, and the second application is a serious game for cultural heritage and in particular for museum environments focusing on the younger visitors.
1. Introduction
Learning institutions such as archives, libraries, and museums are playing a pivotal role in the preservation and distribution of cultural knowledge in our society. In order to ensure our cultural heritage is preserved in an accessible form, such institutions need to embrace new technologies for digital acquisition, content management (including cataloguing), and subsequent exhibition to the public and interaction with this digital heritage by the public.
In recent years, 3D and virtual reality have emerged as areas of extreme interest as methods for visualizing a museum’s digital artifacts in different context, and particularly over the Internet [1, 2]. This has led to more immersive, engaging, and interactive exhibits and experiences. Virtual museums are valuable to the end-user for efficient and remote learning about their local heritage in a diverse multimodal manner [3] allowing them to interact through many senses to the knowledge representations and artifacts. In this way, museums as a response to changing requirements from users, including pervasive use of mobile devices and the Internet, are changing their way of presenting information about their cultural artifacts to the public.
Modern cultural heritage exhibitions have evolved from static exhibitions to dynamic and challenging multimedia explorations. The main factor for this has been the emergence of the world-wide web, which allows museums and other heritage exhibitions to be presented and promoted online. Museum virtual environments can offer much more than static websites, which typically provide little more than a catalogue of pictures and text. Digital artifacts or cultural objects can be presented in a virtual museum and viewed in forms that offer the user multiple degrees of freedom. Learning institutions, and particularly museums, have been able to digitize their collections with relative ease through the use of simple photo imaging and photogrammetry [4]. A virtual museum can consist of many exhibitions representing different museum collections. Additionally, users can select some cultural objects and observe their digital representations in the context of real artifacts in an augmented reality (AR) scene. Museums are particularly interested in finding new ways to deploy engaging and stimulating exhibitions, within their establishment to foster a wider appreciation and understanding of cultural heritage, particularly amongst younger generations of “digital natives” [5].
Virtual museums that exploit both VR and AR can offer much more interesting interactive scenarios for the viewing of digital collections such as virtual reconstructions of their actual artifacts adding to historical realism and providing real-world context VR interfaces, and interaction techniques can offer many advantages for museum visitors. For example, many interaction devices are now available that can be integrated into multimodal virtual and augmented reality interactive interfaces [6]. These technologies can be integrated with low-cost PC desktop systems as well as bespoke museum kiosks. A major advantage of using desktop PCs is that they can be easily repurposed for inclusion in new virtual exhibitions.
To stimulate engagement amongst audiences, it has been suggested that the use of Web3D, VR, or AR in virtual museum exhibitions or tours does not reach its fullest potential through simply present virtual objects and descriptions, that is, a 3D replacement to the traditional museum website; rather, elements such as narrative can strongly reinforce the visitors learning and understanding of cultural content [7]. Museums are an ideal site at which to exploit these virtual technologies, offering challenging research opportunities whilst providing novel ways to present regional or national heritage. Furthermore, they can offer new consultation methods for archaeological or cultural sites.
2. Virtual Museums and Serious Games
Over the last decade, various kinds of virtual museums have been developed either in the museums’ own physical environment or over the Internet, through both static and more interactive media such as serious (educational) games. Virtual museums offer the potential to immerse a visitor in an interactive environment, using telepresence to allow greater reach as well as the freedom to interact closely with exhibits.
One project seeking to capitalize on this potential is Agamemnon. This project aims to exploit the recent advancements in smartphone technology with improved imaging, connectivity, and multimedia capabilities to create innovative tools that will be effectively used by the learning institutions. These tools will allow visitors of either museum galleries or archaeological sites to establish their preferences in terms of interests and the level of information that they wish to acquire about artifacts. In turn, Agamemnon will create a route through the site to include the most interested points based on the visitor profile and time constraints. Additionally, the visitor will be allowed to interact with the system by taking pictures of artifacts that they are interested in and receive information about them [8, 9]. The information will be in the form of 3D reconstructions, images, movies, textual information, and so forth (see Figure 1).
Another example of a virtual museum is the Kivotos. Kivotos is a virtual reality environment based on a CAVE system. The visitors of the museum are participating in the virtual environment by wearing stereoscopic 3D glasses and interacting with the environment by using a magic wand (see Figure 2).
Another example of a VR installation in museums is at the Melbourne Museum in the Science and Life Gallery in Australia called virtual room. The virtual room consists of eight screens placed at 360 degrees (see Figure 3).
ART-E-FACT project proposed that the use of semantic web services can enable the learning institutions to make their cultural content available to researchers, curators, or public in a meaningful way [12]. Furthermore, the consortium acknowledged the fact that the use of digital storytelling and mixed reality technologies will create a new dimension for artistic expression. The project created a platform for developing and exploring new forms of creation, presentation, and interaction that allows the artist to create artistic expressions in an innovative way [13]. The system is composed of a story telling engine that controls the virtual environment with virtual characters that interactively present a work of art, the rendering engine, the authoring tool which defines the story (presentation), the database that contains information about the cultural objects, and a gesture recognition system that detects hand gestures and infrared markers on dedicated objects. Furthermore, the project provides tools for 3D content generation. The ART-E-FACT project offers many advantages to the learning institutions [14]. Firstly, it allows the user to discover information about the artifact that he or she would never be able to know by simply reading the labels with little or no information about the artifact. Secondly, with the use of virtual reality (VR) and augmented reality (AR), the learning institution can cost effective engagement with their visitors and increase the visitor’s presence. Additionally, the system offers a hand gesture recognition system in order to allow the user to interact with the virtual artifacts. However, the implementation of such an interface is not feasible in a learning institution because the user has to be trained in order to use the system [15].
(Augmented representation of cultural objects) ARCO project is an EU funded project aimed at developing technology for museums to create 3D virtual exhibitions on the web. The project is based on the field of cultural heritage and archaeology. The ARCO project provides a set of tools for digitization, management, and presentation of heritage artifacts in virtual exhibitions [16]. In order to evaluate the system, ARCO collaborated with two museums the Sussex Archaeological Society [17] and the Victoria and Albert Museum [18]. Several different virtual museum exhibitions were created in order to evaluate the ARCO system. The two most memorable presentations (or virtual museum exhibitions) due to the historical significance were those focused on the Anne of Cleves [19] Museum and the Fishbourne Roman Palace [20] (see Figure 4).
Another typical example of engaging visitors to learn about ancient Egyptian history is the virtual Egyptian temple. The game is based on a hypothetical virtual Egyptian temple [21], which has no real-world equivalent. This temple embodies all the key features of a typical New Kingdom period Egyptian temple in a manner that an untrained audience can understand. The temple has four major areas, and each one has an instance of the High Priest, a pedagogical agent. Each area of the virtual environment represents a different feature of the architecture of that era. The game engine that this system is based on is the Unreal Engine 2 (Figure 5) [21]. The objective of the game is to explore the model and gather enough information to answer the questions asked by the priest (pedagogical agent).
The Roma Nova project is based upon the Rome Reborn model of ancient Rome 340 AD. The game is built upon Rome Reborn [22], the most high-fidelity model of Ancient Rome currently in existence, providing a 3D digital model which may be explored in real-time. Rome Reborn includes hundreds of buildings which are procedurally generated based on accurate historical knowledge, 32 of which are highly detailed monuments reconstructed based on accurate archaeological data (see Figure 6).
The aim of the project is to provide a distributed tutoring environment for children 11–14 year olds to support cross-disciplinary study, as part of an exploration and social interactive learning model [23]. A particular emphasis of this project is the benefit virtual characters and game-based features can bring to learning environments, going beyond reconstructions of architecture and towards a more holistic approach including avatars and direct interaction by the user. The project is also exploring the use of the semantic web as a basis for character dialogues, effectively allowing the user to opaquely search the web through conversations with virtual characters, who are endowed with knowledge of their surroundings through geocoding and semantic filtering.
AR technologies can be combined with existing game engine subsystems to create AR game engines [24] for the development of AR games. AR has been applied successfully to gaming in cultural heritage. One of the earliest examples is the Virtual Showcase [25] which is an AR display device that has the same form factor as a real showcase traditionally used for museum exhibits and can be used for gaming. The potentials of AR interfaces in museum environments and other cultural heritage institutions [26] as well as outdoor heritage sites [27] have been also briefly explored for potential educational applications. More specific gaming examples are the MAGIC and TROC systems [28] which were based on a study of the tasks of archaeological fieldwork, interviews, and observations in Alexandria. This takes the form of a mobile game in which the players discover archaeological objects while moving. Another cultural heritage AR application is the serious game SUA that was part of the BIDAIATZERA project [29]. This project takes the form of a play which recreates the 1813 battle between the English and the French in San Sebastian. Researchers developed an interactive system based on AR and VR technologies for recreational and educational applications with tourist, cultural, and socioeconomical contents, the prototype for which was presented at the Museo del Monte Urgull in San Sebastian.
Having reviewed a range of projects in the area, the next section goes on to describe our specific case study: the Herbert Virtual Museum. In this project, we describe the creation of a game-based approach to conveying cultural heritage set within a virtual reconstruction of a real-world museum.
3. Herbert Museum Virtual Museum
The Herbert Virtual Museum aims to create an interactive relationship between the visitors of the Herbert Museums and Galleries via the use of mobile applications. The project was funded by the MLA Renaissance in the Regions programme. The Herbert Virtual Museum application was developed by the Serious Games Institute and Coventry University in collaboration with the Herbert Art Gallery. The Herbert Virtual Museum application was developed for both Apple iPhone and Google Android mobile platforms.
The Virtual Museum, known as the “Gallery,” provides content in the form of the museum’s collections and galleries, as well as the content creation skills of staff in terms of digital photography, filmmaking, and authoring. The Serious Games Institute brought the programming and technical expertise required to create a coherent virtual world for this content, as well as an underlying gameplay model for the game described in Section 4.
The application developed includes an interactive interface and digital map of the entire museum, allowing visitors to navigate around the galleries. The system also offered a range of virtual guided tours for the individual user. The content provided to the user includes a range of audio, text, photos, images, and videos. Quick response (QR) codes were recognized as a useful tool to engage the user and add an extra dimension to the app. Each object within the museum and art gallery has a dedicated QR code. A reader built in to the application allows the user to capture the code, and the relevant information is immediately presented to the user. The dedicated content management system allows the museum to dynamically update the content and generate the QR codes independently (see Figure 7).
Social network links have also been developed allowing visitors to update their status on Facebook and Twitter in addition to blog entries on Tumblr. The update informs friends/followers about the particular items and objects currently being viewed by the user, generated automatically by the app.
As an addition to the app, a fixed installation within the museum space was created, called the “Roll of the Fallen.” This is an interactive touch screen that features floating poppies. When a poppy is selected a bio appears as a tribute to those who lost their lives during the blitz of World War II. The museum provides a facility whereby new biographies can be submitted by members of the public.
The mobile apps for The Herbert Gallery were all released simultaneously and are freely available for download to the general public as of 1 April, 2011. The app is currently available for Android, iPad, iPod, and iPhone platforms (see Figure 8).
4. Priory Undercroft: The Game
The priory undercroft game is based on the virtual reconstruction based on the priory undercrofts remains of Coventry’s original Benedictine monastery, dissolved by Henry VIII. The aim of the game is to provide an insight on the daily activities of the monks living on the monastery and to raise the interest of the younger generation to cultural heritage. The player interacts with various nonplayer characters in the 3D environment, and they have to uncover clues that will help them to solve the puzzles. Each time a new puzzle is solved, the player is prompted to answer a question related to the history of the site in order to progress to the next puzzle [30]. The first version of the games was based on QUEST3D (see Figure 9). The initial scenario was linear, and the player had to follow the clues in order to complete the game. The game records decisions and actions made by the user throughout a simulator session for later interrogation by the educator.
Table 1 gives an example of puzzles that the learner has to solve to progress to the next puzzle.
The second iteration of the game was based on unity. However, in this version the player interacts with NPC in order to uncover clues and solves the puzzles. The interaction with NPC was based on the level-of-interaction (LoI) framework, codeveloped by the Serious Games Institute and Toulouse University.
The system architecture, which is presented in Figure 10, is composed by three modules including(i)the visualization module,(ii)the content management module, and(iii)the artificial intelligent module (LOI).
The content management system (CMS) is implemented within the authoring tool of UNITY3D and allows the educator to create various learning materials. The CMS is also responsible for the organization of tasks that the user will complete during a particular scenario. Tasks are organized according to a preplanned decision tree (Figure 11).
Each scenario step may contain one or more decision sets. Each decision set is connected to the next scenario step by a transition that occurs only when all decisions of a given decision set has been taken. In some cases, a decision set may contain a single decision. In this case, the decision leads to the next scenario step.
A decision tree strikes the balance between “user flexibility” and “writer flexibility” [31]. The learner is guided through the scenario with a series of puzzles which they have to solve in order to progress to the next puzzle and finish the games.
The LoI framework simplifies the interaction between the player and the nonplayer characters (NPCs). Graphically, the LoI can be represented as auras [11] of increasing complexity centered on the player’s avatar (see Figure 12) and based on a simple social space metric [23] and is divided to three levels. The first level aims to populate the characters with authentic crowd in order to increase the immersion of the player. Characters located in closer surrounding of the player belong to the interaction level. Finally, a character inside the dialogue level interacts with the player in a natural way, ultimately using speech recognition and synthesis. All the NPCs by default belong to the background level, but as the player moves on the environment, they happen to get closer or away from the player and thus enter or exit the interaction or dialogue levels.
Next, to acquire initial feedback on the game, a self-contained executable file was supplied to 5 Coventry University students based on the hallway usability testing methodology. The intention of the evaluation was to gather information on the playability and enjoyability of the game and to discover potential technical problems. All of the users had some experience with games, and a few of those involved also had experience with games programming. For all users, the aim of the game was presented, and it was explained that the players should not expect a complete game, but rather a prototype.
The average time of the tests was approximately 30 minutes. They were asked to talk through what they were doing and how they felt as they played the game. Overall, recorded feedback was very encouraging, and all users agreed that the game has a lot of potential for cultural heritage applications. They also mentioned that they prefer the idea of “playing” and “learning” at the same time. All users agreed that the educational aspect of the game is obvious and helps them to understand and learn something about the history of priory undercrofts.
5. Conclusions
Virtual museums can utilize new technologies in order to enhance the user experience. The benefits of such technologies are noteworthy to museum curators, students, and visitors. Virtual museums have the potential to preserve and propagate their cultural information through the use of such innovative tools, by engaging the visitors and making their experience more interactive and immersive, allowing them to relive the past more accurately and extending the experience of the museum visit beyond the period of the visit. Virtual museums are an engaging way to support better learning of the past and to bring alive ancient artifacts with great appeal to visitors and can promote real sites. The two new projects are increasing participation of new audiences and extending the scope and reach of the museum into digital realms, through gaming and simulation technologies.
Our case study within the Herbert Museum shows the wide range of ways in which technology can be appropriated to enhance visitor experiences. Furthermore, it demonstrates how this can go beyond simple exhibits: elements such as the “Roll of the Fallen” and undercroft game would not be possible without the use of digital media and displays. This reinforces the other examples in the background: digitization of exhibits and artifacts is a first step in an exploration which goes beyond simply allowing the visitor to view the item remotely, instead embedding in the experiences, cultures, and emotions of the past. Furthermore, as the “Roll of the Fallen” demonstrates, this can include living histories, allowing visitors to record and communicate their own histories for future generations.