An Empirical Study of Visitors’ Experience at Kuching Orchid Garden with Mobile Guide Application

Othman, Mohd Kamal; Idris, Khairul Izham; Aman, Shaziti; Talwar, Prashanth

doi:https://doi.org/10.1155/2018/5740520

Advances in Human-Computer Interaction

On this page

Abstract Introduction Materials Results Conclusions Conflicts of Interest References Copyright Related Articles

Research Article | Open Access

Volume 2018 | Article ID 5740520 | https://doi.org/10.1155/2018/5740520

An Empirical Study of Visitors’ Experience at Kuching Orchid Garden with Mobile Guide Application

Mohd Kamal Othman,¹Khairul Izham Idris,¹Shaziti Aman,¹and Prashanth Talwar¹

Academic Editor: Francesco Bellotti

Received05 Jul 2017

Revised06 Feb 2018

Accepted02 Apr 2018

Published03 Jun 2018

Abstract

This empirical study was conducted to measure visitors’ experiences with a mobile guide application at Kuching Orchid Garden (KOG). A between-group experimental design with 114 participants was conducted to test three groups; a group using the mobile guide application as an information aid, a control group (with no information aid), and a group using pamphlets to explore the KOG. The Museum Experience Scale (MES) was used to evaluate visitors’ experience for all participants, whilst the Multimedia Guide Scale (MMGS) was used to evaluate the visitors’ experience with the mobile guide group. The most notable result from the Museum Experience Scale (MES) showed an impact on the visitors in terms of knowledge and learning when using the mobile guide application. However, the study found that enhancing visitors experience goes beyond simply providing interactive technologies in public settings to aid with information delivery. A limitation was providing relevant information in a timely and seamless manner due to inaccuracies of mapping between physical and digital environments. Future works should consider beacons and other Bluetooth low energy (BLE) technology to address the issues with location based devices. It is also important to highlight that the use of one’s own device had a significant impact on learnability and control of the device, thus suggesting that the BYOD concept should be widely used in informal educational settings implementing mobile guide applications. The use of MES and MMGS informs future researches with an understanding of the different dimensions of visitors’ experiences with mobile guide technology in public spaces to inform mobile application development that may further boost visitors’ engagement, emotional connection, and meaningful experience.

1. Introduction

“Dewey’s two aspects of the quality experience can be restated as follows: the visitor interacts with the exhibit and has an experience, and the visitor assimilates the experience so that later experiences are affected” [1, p. 36]. Mobile guide applications in public spaces offer visitors a different approach of engagement and experience [2–4]. The Ansbacher notion on visitors’ experiences motivated this study to look at the impact of mobile guide technology on visitors’ experience at Kuching Orchid Garden (KOG).

This current research aims to design, develop, and evaluate a mobile guide application by comparing visitors’ experience at Kuching Orchid Garden (KOG) with and without the aid of the mobile guide application. The mobile guide application for Kuching Orchid Garden (KOG) uses the free-choice learning approach in which visitors have the freedom to choose the content they want to learn, thus emulating the real visiting experience at Kuching Orchid Garden (KOG). This study also employed the Bring-Your-Own-Device (BYOD) model to eliminate familiarizing users with the technical workings of a new device as they would use their own devices, thus easing the users’ interaction with the mobile guide application.

Kuching Orchid Garden (KOG) is a 15.4-acre property housing a wide collection of orchids in Borneo. This orchid garden houses a total of 75,000 plants comprising 85 genera. KOG is divided into two sections, a nursery, and a display area and it is open to visitors every day except Monday. Admission to the KOG is free. Currently, KOG does not provide enough support to the visitors. Despite having vast resources about orchids, many visitors had difficulties in learning about the genera on display at KOG due to the lack of information delivery. Hence visitors can be seen going through the orchids with no particular purpose. This could be explained by a study conducted by Templeton [5] who found that many visitors were not engaged or were disconnected when there were lingering unanswered queries as they went through the entire museum exhibition. The use of an interactive mobile guide application can possibly alleviate the issue by providing real-time information to the visitors.

Visitors’ experience with the mobile guide application was evaluated using Multimedia Guide Scale (MMGS) and Museum Experience Scale (MES) developed by Othman et al. [6]. Petrie et al. [7] successfully conducted a few studies to measure visitors’ experiences at historical churches with mobile guides and this has contributed to the understanding of different types of visitors’ experiences. They also noted that visitors to public spaces varied and expected different things when they visited the places, be it with friends, companions, family members or even with school trips. Hence, this study was conducted to measure the impact of using a mobile technology (mobile guide) at a public space particularly at Kuching Orchid Garden (KOG).

The following were the key hypotheses investigated as part of this study:(1)The level of knowledge and learning improves with the use of mobile guide at Kuching Orchid Garden (KOG).(2)The visitors’ engagement improves with the use of mobile guide at Kuching Orchid Garden (KOG).(3)The visitors’ emotional connection improves with the use of mobile guide at Kuching Orchid Garden (KOG).(4)The visitors’ emotional experience improves with the use of mobile guide at Kuching Orchid Garden (KOG).

It is important to highlight that the mobile guide application developed for this study is only suitable for android platform users as 81% of smartphone users in Malaysia are android users. In addition, other features such as RFID, Bluetooth, GPS, and other related features on the mobile guide were not utilized and discussed. This research article only focuses on users’ experiences with the mobile guide application as an information aid at Kuching orchid Garden as opposed to the paper-based pamphlet and no guide users.

1.1. Technologies for Informal Educational Settings

The evolution of technologies has enabled the use of a variety of technologies in informal educational settings to enhance visitor experience such as 3D visualizations, wall projection displays, tablets, and mobile devices. It cannot be denied that these various technologies have a significant impact on their visitors, but it is important to understand how technologies can be used to improve the visitor experience at such places. Recently, Zimmerman and Land [8] addressed the use of mobile devices for informal science education at public spaces and provided four suggestions: support social interaction within informal settings; enhance the visitors’ experiences through the use of games, scientific narrative, and disciplinary-relevant aspects; integrate activities to avoid visitors becoming passive learners; provide after-visit support to bridge the learning using social media or other relevant media. Previous studies have highlighted the use of mobile guide technologies at various informal educational settings such as museums [3, 9–16] botanic gardens [2, 17]; zoos [18]; and aquariums [10, 19–21].

Though public space institutions are keen to adopt the newest available technology to attract more visitors, several studies showed that, in the early years of the adoption of mobile technologies in public space, it has failed to improve visitors’ experience [22, 23]. Pekarik [24] highlighted that the visitors’ experiences at public spaces, particularly cultural heritage sites, were extremely diverse, and providing the technology might not be the optimal solution. It is important to know about the visitors’ engagement with the exhibits on displays or if they spent too much time understanding the technologies used. For example, the Personal Digital Assistant (PDA) was adopted to enhance the learning experience of museum visitors. However, visitors stumbled into situations that required them to troubleshoot the device because they were unfamiliar with it and this might affect the learning experience [9]. In addition, it was also found that the device put visitors in a passive “curator mode” because they had no freedom to explore the museum at their own will. This in turn resulted in low scores of visitors’ experience, as they were not fully engaged with the museum artefacts.

The reason why previous use of mobile technologies in public spaces was not very successful was in part due to the lack of an understanding of visitor needs and experiences. This was highlighted by Sharples et al. (2010) who pointed out that “the main barriers to developing new modes of mobile learning are not just technical but social” (Sharples et al., 2010; p. 4). Brown [25] shared a similar view and stated that the most important aspect of mobile learning was that learners were constantly on the move whilst accessing information. It differs significantly from traditional instructional design in that it is more of a learner-controlled environment. Burston [26] highlighted that one of the main reasons the failure in the adoption of mobile technology in learning is the pedagogical innovation in which it did not fully utilize the mobile device affordance. Therefore, the design of the mobile guide application should follow the “free learning concept” in which the visitors can choose which content they would like to view. This allows visitors to actively construct their experience in more meaningful ways in accordance with their needs.

A major benefit of the use of technology is that it can deliver text, images, audio, video, multimedia, and other types of information visualization to the users. This helps users to gather more information on what they are learning in formal and informal educational settings. However, it is also important to highlight issues with the amount of information provided particularly on the mobile devices. In the context of informal educational settings, too much information could possibly lead to “cognitive overload” and users can become “lost in hyperspace” [27]. Therefore, personalization of users’ learning experiences to suit different user needs is important. A study by Walker [17] highlighted the importance of accommodating different user needs in informal learning settings such as historical sites, botanic gardens, and museums.

In addition, the question as to whether visitors at such places (informal educational settings) should use their own devices whilst viewing the information about the exhibition and its’ artefacts surfaced due to the time spent by visitors to understand the technologies provided on site. This begets the concept of Bring Your Own Device (BYOD) where individuals are to bring their own Internet-enabled device to aid them with their work and learning. The BYOD model has been widely used (i.e., [28–32]). BYOD would make the visitors spend less time familiarizing themselves with the workings of the device as they are using their own devices.

1.2. Personalization Concepts and Bring Your Own Device (BYOD)

Personalization has been around before the advent of Internet technologies and was mainly focused on the personalization of computer interfaces. Currently, the personalization concept is widely used for different purposes, for example, e-commerce applications, tourism, education, finance, culture, health, and many other related areas [33–35]. Informal educational settings such as museums have started to adopt this concept for their virtual museums as well as their guides at the exhibition galleries to improve the information delivery to their visitors. Personalization could ease the users’ visit [36] by filtering information according to their needs, facilitating navigation and information access [34].

In the context of mobile guide for informal educational settings, the personalization concept should not be limited to the contents of the guides but can also include personalization of devices. For example, visitors may bring their own mobile devices (BYOD) which allows them to personalize the interface to access the information, such as viewing it on a bigger screen with a tablet or phablet or even changing the font or colour combination of the text. BYOD also helps to save cost for museums and other informal educational settings and their visitors because the institutions do not have to buy any devices or systems and the visitors do not need to pay extra rental of the devices [37]. The organizations also do not have to pay for manning or maintenance of the devices and systems.

1.3. User Experience (UX)

Garrett [38] defined UX as the experience created by a product for the person who uses it (user) whilst Hassenzahl and Tractinsky [39] believed that the phenomenon of UX consists of several key elements which are emotion and effect, the experiential and beyond the instrumental. Experience is a concept that consists of meaning and history to an individual [39–41], whilst according to Bell et al. [10], Falk and Dierking [42], and Falk [20] it is the result of individuals who are engaged in the process of meaning-making. Therefore, UX is often related to the concept of engagement.

The engagement concept was highlighted as one of the qualities of UX with mobile device [4, 6], whilst Schaeffer and Polgreen [43] valued engagement as the interaction between visitors or users and the content provided in the mobile application. O’Brien and Toms [44] discussed how the UX could be evaluated using the user engagement components. They further explained that the quality of user experience was based on a positive interaction with the computer which involved the relationship between the user engagement and the UX based on various experiential attributes such as aesthetics, novelty, involvement, perceived usability, and endurability. Recently, Pallud [45] discussed the use of interactive technologies in a French museum to engage their audience and promote a positive learning experience. Results indicated that ease of use and interactivity could influence the emotional process (authenticity and cognitive engagement) which in turn could influence learning.

Several studies have looked into measuring engagement and visitor experience. For example, Lykke and Jantzen [46] developed 10 dimensions of experience and evaluated them at the Center for Art and Media (ZKM), Karlsruhe (Germany). The ten dimensions of experience were as follows: involving, spontaneous, interesting, relevant, learning, unique, interactive, fun, close, and (10) authentic. They found that experiences such as interesting, interactive, relevance, and involving were more frequently expressed. Other studies (for example, [3, 6]) used three components in measuring engagement with mobile guide technologies at informal learning settings particularly museum and historic churches: general usability, quality of interaction, and (3) learnability and control. Although learnability was usually associated with general usability, a previous study by Othman et. al. [3, 6] showed that it formed a different component with mobile guide technology. Quality of interaction is an important measure in evaluating engagement as it was posited that visitors have more meaningful learning or experience [47–50] without the trouble of learning to manage a technology, device, or an application. Learnability and control is a component that takes into consideration the visitors’ capabilities to learn and control the application without any prior knowledge. The familiarity of users with their own device cancelled out the factors of learning to manage new devices [2, 51]. Thus, these components are essential in evaluating and improving visitors’ engagement in public space.

2. Methodology

This experimental design study was conducted in the wild at Kuching Orchid Garden (KOG), a 15.4-acre property housing 75,000 plants comprising nearly 82 genera of orchids in Borneo. Participants were randomly assigned into three groups: control group (no aids/guide), paper-based pamphlet, and mobile guide. The research was performed with the aid of pamphlet and mobile guide application and designed and developed by the researchers with the information provided by the KOG management. The mobile guide application, which was developed, followed the free-choice version which enabled visitors to select points of interest during their visit. Although there were 82 genera of orchids at KOG, only 35 genera were available for visitors during the period of this study.

3. Participants

As the research was conducted in the wild, which meant that the research was done in situ, participants were recruited from the available pool of visitors to KOG. A total of 114 participants of which 60 were males and 54 were females took part in this study over a one-month period. Convenience sampling technique was used in recruiting the participants. Participants were randomly assigned to groups. The participants varied from local tourists to tourists from various parts of the world such as France, Germany, Russia, and Switzerland. Among the participants, 53 were native speakers of English.

The participants were divided into three groups: control group (no aids/guide), paper-based pamphlet, and mobile guide as illustrated in Table 1.

Participants who used the mobile application were further categorized into two groups, which included native and nonnative speakers of the English language. It is important to highlight this aspect because the mobile application guide is in English and English is not the first language for the local participants. Hence, it might affect the overall experience, especially the knowledge and learning components.

4. Materials and Equipment

4.1. Mobile Guide Application

The mobile guide application was developed by implementing the mobile application development life cycle (MADL) that included requirement analysis, designing the application, implementation (development), testing, deployment, and maintenance [52]. During a visit to KOG, it was obvious that there was no information provided about the orchids, thus making it difficult for visitors to get more information about the orchids. As most visitors had their mobile devices with them, this presented an opportunity for the use of a mobile guide application to be designed to deliver information about the orchids.

In the requirement stage, it was determined based on visitors’ behavior that the design of the mobile guide application should follow the free learning concept. Visitors preferred to choose which orchids they would like to view as they moved randomly in the garden instead of following a set path. This allowed visitors to actively construct their experience in a more meaningful way according to their needs. Information was to be made available corresponding to the location of the orchids within the KOG.

The design process started with different interfaces sketches to form a storyboard. This process continued with the development of low fidelity prototypes before it was coded into a software application builder (Corona SDK) using LUA script. The application was tested using emulators followed by the actual device. The mobile guide application was further evaluated with the real users at KOG to gain more insights. The final product of the mobile guide application can be seen in Figures 1–5.

Figure 1 depicts the first page of the mobile guide when a visitor accesses the mobile guide application. Then, the visitor will be directed to the main menu as illustrated in Figure 2 and subsequently to the map of the KOG which contains point of interest markers (in red colours) on which users can tap as shown in Figure 3. These points of interest markers then lead to screens that have photographs of the orchids on the left and a genera name on the right, as illustrated in Figure 4.

Figure 5 portrays the screen for the orchid genera, with the beginning of the associated text visible.

4.2. Paper-Based Pamphlet Design

Paper-based pamphlets for KOG were designed using Microsoft Publisher. Figure 6 shows the information provided on the front page of the pamphlet, whilst Figure 7 shows the information on the back page. Care was taken to make sure that information and graphics in the pamphlet are the same as the mobile guide application to avoid any biases between the two information presentation modes.

4.3. Instruments

Participants who visited the Kuching Orchids Garden were requested to complete two questionnaires. The Museum Experience Scale (MES) consists of 37 items with a 5-point Likert scale rating. The scale was developed by Othman et al. [6] and comprises four factors for measuring visitors’ experiences at museum and other cultural spaces. The factors of the MES are as follows:(i)Engagement (MES-Engagement), which refers to the engagement with the exhibitions and exhibits(ii)Knowledge/Learning (MES-Knowledge and Learning), which refers to the knowledge gained and understanding from the information provided(iii)Emotional Connection (MES-Emotional Connection), which refers to the emotional attachment with the contents and context of the exhibits/exhibitions(iv)Meaningful Experience (MES-Meaningful Experience), which refers to the quality of interaction with other visitors and the exhibits/exhibitions.

Visitors were also asked to complete a Multimedia Guide Scale (MMGS) also developed by Othman et al. [6] which contains 20 items with a 5-point Likert scale rating to evaluate the usability of the mobile application. The factors of MMGS are as a follows:(i)General Usability, which refers to the usability of the mobile guide application, whether its functionality is appropriate(ii)Learnability and Control, which refers to the ease of use of the guide(iii)Quality of Interaction, which refers to the usability of the guide. Although it is often associated with usability component, a separate component was formed within the mobile guide for cultural heritage sites and museum.

4.4. Instruments Selection

It is important to highlight that the rationale of using MMGS and MES in this study is because Orchid Garden is considered as an informal learning setting like museums and other cultural heritage sites. The evolving definition of museums by the International Councils of Museum (ICOM) includes botanic gardens, zoos, aquariums, and science centres. Moreover, previous studies (i.e., [17, 53, 54]) have also discussed their studies within similar contexts.

The instruments have been used to measure visitors’ experiences in different museums and historical churches in UK [3, 6, 7], Korea [55], Malaysia [56], China [57], and Austria [58, 59]. In addition, previous studies such as Baker et al. [60]; Kabassi [61]; Konstantakis et al. [62]; Moesgaard et al. [63] also highlighted the instruments development to measure visitors’ experiences. The current study is part of study conducted by Idris [64]. Furthermore, one of the authors was involved in the development of the instruments used in this study. The Cronbach alpha for both instruments MES and MMGS was 0.94 which is considered highly reliable [4]. Details of the instruments items are illustrated in Tables 2 and 3.

4.5. Procedure

The present study involved three groups of participants; (i) visitors equipped with the mobile guide; (ii) visitors without any guide; and (iii) visitors with paper-based pamphlets as information guide. The research procedure was as follows:(1)Visitors to the KOG were approached during a one-month duration in April 2016, (during operating hours) and asked if they would like to participate in this study.(2)They were given a brief description of the study relating to visitors’ experiences at KOG, with and without the mobile guide technology.(3)Visitors who agreed to participate were informed about their right to withdraw from the study at any time without prejudice.(4)Subsequently, they were asked to read and sign the consent form and were provided with the research instruction.(5)Visitors were randomly assigned to the three groups.(6)Visitors in the mobile guide group were requested to install the KOG mobile guide application on their mobile phones.(7)Visitors then commenced their journey through KOG.(8)All visitors were asked to complete the questionnaire (MES) after their visit.(9)Visitors in the mobile group were asked to complete another set of questionnaire (MMGS) after they completed the MES.(10)After a short debriefing session, the researchers answered any queries by the visitors.

5. Results

5.1. Museum Experience Scale (MES-Engagement)

As illustrated in Table 4, statistically significant differences were noted between the groups as determined by one-way ANOVA , , , thus, rejecting our hypothesis.

A small effect size of .14 was observed between the groups. In addition, a post hoc Tukey test was also carried out to have a better understanding on whether there were any statistical significant differences between the groups (modes of visit). The Tukey post hoc test revealed that the mean score for the mobile guide group (, SD = 0.587) was not statistically significant different compared to the no guide group (, SD = 0.647). On the other hand, there were statistically significant differences between mobile guide and paper-based pamphlet group (, SD = 0.348). Statistically significant difference was also noted between paper-based pamphlet and no guide group. In conclusion, although the no guide group showed a higher mean ranking compared to the mobile guide group, the differences were not statistically significant on the visitors’ engagement.

5.2. Museum Experience Scale (MES-Knowledge and Learning)

The result of the one-way ANOVA showed that there was a significant difference between the modes of visit , , for the Knowledge/Learning component, thus, rejecting our hypothesis. A small effect size of .11 was observed between the groups. In other words, the magnitude of effect accounted for 11%. In addition, a post hoc Tukey test was also carried out to have a better understanding on whether there were any statistical significant differences between the groups (modes of visit). Results indicated that there was no statistically significant difference between mobile guide (, SD = 0.508) and paper-based pamphlet groups (, SD = 0.335). On the other hand, there was a statistically significant difference between the mobile guide and no guide groups (, SD = 0.593), as well as between the no guide and paper-based pamphlet groups (, SD = 0.342). Hence, it showed that the use of mobile guide and paper-based pamphlet had a greater impact on the visitors’ learning process at KOG.

5.3. Museum Experience Scale (MES-Emotional Connection)

The result from the one-way ANOVA indicated that there was a statistical significant difference between the modes of visit on Emotional Connection , , , thus, rejecting our hypothesis.

A medium effect size of .23 was observed between the groups. Variance explained was 23% for the modes of visit. In addition, a post hoc Tukey test was also carried out to have a better understanding on whether there were any statistical significant differences between the groups (modes of visit) and the result revealed that there was no statistically significant difference between mobile guide (, SD = 0.583) and no guide groups (, SD = 0.411). However, there was a statistically significant difference between paper-based pamphlet (, SD = 0.342) and mobile guide groups. In addition, it was also identified that there was a statistically significant difference between paper-based pamphlet and no guide groups. In conclusion, although the no guide group had the highest mean score, the difference was not significant enough to be considered as an apparent outcome.

5.4. Museum Experience Scale (MES-Meaningful Experience)

Further analysis of the meaningful experiences showed that there was a statistically significant difference between groups as determined by one-way ANOVA , , , thus, rejecting our hypothesis. Eta revealed a medium effect size of 0.22 for the two groups which also stipulated a 22% variance. In addition, a post hoc Tukey test was also carried out to have a better understanding on whether there were any significant statistical differences between the groups (modes of visit). The result indicated that there was a significant difference between mobile guide (3, SD = 0.338) and paper-based pamphlet groups (, SD = 0.377) and also that there was a significant difference between paper-based pamphlet and no guide groups (, SD = 0.474). However, there was no statistically significant difference between mobile guide and no guide groups.

5.5. Multimedia Guide Scale (MMGS)

Figure 8 shows that there were differences in the mean scores of the components in MMGS (General Usability, Quality of Interaction, and Learnability and Control). Figure 8 also indicates that native speakers of English language had a higher mean score on all three components compared to nonnative speakers of English language. The component of General Usability had a mean score (, SD = 0.253), Quality of Interaction (, SD = 0.385), and Learnability and Control (, SD = 0.181). To investigate whether the differences were significant, further analysis was conducted using one-way ANOVA.

The result from the one-way ANOVA showed that there were statistically no significant differences in the mean score between the native and nonnative English speaker in the component of General Usability , and Quality of Interaction , . On the other hand, there was a statistically significant difference between the native and nonnative English speaker , in the component of Learnability and Control.

6. Discussions

All hypotheses were validated using the MES and MMGS instruments. The four components of the MES were used to compare the visitors’ engagement among three different groups (no guide, paper-based pamphlet, and mobile guide application). The results showed that there was a significant difference between modes of visit at KOG as indicated by each of the MES components in the previous section.

Schaeffer and Polgreen [43] defined engagement as the visitors’ interaction with the contents. Though this aim of the mobile guide was to enhance visitors’ engagement at KOGS, the result (MES-Engagement) showed the opposite whereby the highest mean score for engagement was for visitors with no guide. This could be due to engagement being the direct interaction between the visitors and the orchids (content) at the garden. However, comparison between the mobile guide application and paper-based pamphlet showed that the use of mobile application had a higher mean score on visitors’ engagement. This could be due to the interactive component of mobile guide content as stated by Hart et al. [65] which found that higher effect (engagement) ratings were extracted from the more interactive contents compared to textual content (menu-link navigation). This is further supported by Wakkary et al. [16] who found that using media (e.g., audio, video, or image) would spawn the visitors’ engagement that resulted in the interest of the visitors towards the artefacts. A more recent study concretized the notion that the use of technology influenced the emotional process (authenticity and cognitive engagement) due to the ease-of-use and interactivity factors, which subsequently positively influence learning [45].

The result from the Knowledge/Learning component (MES-Knowledge and Learning) clearly justifies the need for providing information about the orchid displays at KOG. The group that did not use any guide scored the lowest mean whilst the mobile application group ranked the highest in Knowledge/Learning. This is obviously due to the lack of information about the orchids displayed at the garden. The mobile guide provided information that enabled visitors to acquire knowledge about the orchid displays, thus enhancing the Knowledge/Learning component. However, the mode of information delivery played a significant role as the paper-based pamphlet group had a lower mean score on the Knowledge/Learning component than the mobile application group though the stimuli (text and graphics) presented in the paper-based pamphlet were the same as in the mobile guide application. This could be due to the way the information was presented to the visitors. The interactivity of information delivery may affect the knowledge/learning experience of the visitors. A study by Findlay [2] found that mobile application gave visitors more opportunity when learning about the content and this was influenced by their needs. This is further supported by Pallud [45] who stated that ease-of-use and interactivity factors positively influenced learning [45]. Hence, informal educational institutions need to find solutions to deliver the information to visitors using the appropriate medium that will benefit both visitors and the informal educational institution.

Like the Engagement component (MES-Engagement), the highest rank in the Emotional Connection component (MES-Emotional Connection) is the group that did not use any guides during their exploration at the garden. This relates back to the Engagement component of the MES (MES-Engagement) that is directly proportional to the Emotional Connection component. Findings by Schaeffer and Polgreen [43] as well as [45] stated that visitors were likely to be engaged when they were emotionally attached. Therefore, the group that did not use any guides scored the highest on both components. This could be due to a few factors as presented by previous studies. A research by Falk and Gillespie [66] found that the external factor such as the layout design might affect emotion, which contributes to the visitors’ engagement at science centres. It could be said that the external environment was more appealing than the interface design of the digital environment in the mobile guide. This could be due to lack of aesthetics resulting in low emotional arousal with regard to the mobile guide environment as compared to the actual surrounding area. Emotional connection is crucial as it can enhance visitors’ engagement, which would then supply the basics for visitors to construct their knowledge on the subject matter of the public space [67]. Therefore, future work must pay attention to the aesthetics of the visual design component of the mobile guide interface design.

One of the aims for visiting informal education institutions and public spaces such as museums, historical sites, and botanical gardens is to achieve a Meaningful Experience A study by Falk et al. [19] showed that 7 percent of zoo or aquarium (public space) visitors were experience seekers. Therefore, this suggested that a meaningful experience might indicate an impactful experience that was beyond information acquisition [68]. The result from this study (MES-Meaningful Experience) shows that the group that used the mobile application came second to the group that did not use any guide in terms of having a meaningful experience. However, the differences were not statistically significant between the two (2) groups. Hence, both groups of visitors had an impactful episode during their visit to KOG. This result echoes Chiodo and Rupp [67] who stated that an exhibit should be more than just knowledge that individuals could acquire in order to have an impact on the visitors [67]. It is important that the design best suits the subject matter and the visitor needs [41, 67]. This may imply that the current design of the mobile guide application is not the best design for the subject matter and visitor needs. Therefore, it is proposed that participatory design be used for future works on the mobile guide application where users are included as cocreators to delve deeper into the design the suits the visitor needs.

The three components of MMGS, General Usability, Quality of Interaction, and Learnability and Control, were used to evaluate the use of the mobile application in this study. The result shows that native English speakers had a higher mean score on the General Usability compared to nonnative speakers. This could be due to the mobile guide application being in English, which could pose comprehension issues for some nonnative speakers that was perceived as a usability issue. However, the difference between the two groups was not statistically significant. The overall result of General Usability shows that the total mean score is on the middle of the verdict. In short, the General Usability of the developed mobile application can still be enhanced to improve the score. Usability is one of the most essential aspects of a mobile application as users need usable products [38]. According to Hart et al. [65] the general usability is also affected by the interface design. In the case of the mobile guide application, the interface for points of interests may not match up exactly with the location of each orchid in the physical environment because it is not a GPS-based application. This localization issue could pose as a usability issue. It is proposed that future design utilized GPS or RFID tags to enable a more accurate mapping between physical and digital environments.

Recently, many informal educational settings have focused on the use of beacons and other Bluetooth low energy (BLE) technology to address the issues with location based devices. Visitors at such places are expecting to receive relevant information about the artefacts in a timely and seamless manner. The use of beacons technology can possibly eliminate the boundary between the physical and digital user experiences whilst visiting such informal educational settings thus improves their meaningful experiences and engagement. In addition, the mobile guide developed in this study can be used as an indicator to address the amount of information required for the exhibition, thus affecting their knowledge and learning. The findings from this study could benefit future researchers who want to implement beacons technology in such settings

The Quality of Interaction between the visitors and the mobile application as an information aid is one of the important measures of user experience (UX) [4]. The MMGS result indicates a good response from the visitors who used the mobile application. However, it also shows that there are still some room for improvements for the application. This relates to the component of Meaningful Experience in the MES. The group that used the mobile application had a relatively high score that was higher than the paper-based pamphlet group but lower in comparison to the group without any guide. Therefore, though Othman [4] stated that visitors would have a more meaningful experience if the technologies were interactive, the Quality of Interaction goes beyond simply providing technologies.

The Learnability and Control component shows the most positive outcome out of the other MMGS’s components. The results show a significant difference between the native and nonnative English speaker whereby native speakers of English found it easier to understand and use. As mentioned before, the level of English proficiency may impede comprehension thus affecting, users’ ability to understand and use the mobile guide application. However, the result from native speakers is a positive indicator that the developed mobile guide application is easy to understand and use. Learnability is an important aspect of user experience as it also affects the quality of interaction [69]. Therefore, a remediation could be to provide dual language selection for visitors using the mobile guide application at KOG.

Othman [4] stated that technology should not be a barrier between visitors and the content of the public space. Research conducted by Bartneck et al. [9] found that visitors who were unfamiliar with mobile guides had a negative experience during their visit. Thus, this research used the concept of Bring Your Own Device (BYOD). This allowed the visitors to install the application on their own device; thus there was no need to familiarize themselves with new devices. Therefore, the user has prior understanding of the control of the device, consequently reducing or eliminating the learning curve. This contributed to the high mean score for both native and nonnative groups.

7. Conclusions

The advancement of today’s technologies such as smartphones, tablets, and phablets has enabled the use of a variety of technologies in informal educational settings to enhance visitor experience, particularly mobile guide applications. Mobile guide applications in public spaces offer visitors a different approach of engagement and experience [2–4]; however it cannot be assumed that the deployment of such technologies will automatically enhance the visitor engagement and overall experience as evidenced by the results of this study.

The most notable result from the Museum Experience Scale (MES) used to investigate the overall visitors’ experience showed an impact on the visitors in terms of knowledge and learning when using the mobile guide application. The MES result shows that the visitors’ engagement with the use of the mobile guide application has a higher mean score and a significant difference in terms of knowledge or learning compared to those who have either used the paper-based pamphlet or did not use any guide. However, the study found that enhancing visitors experience goes beyond simply providing interactive technologies in public settings to aid with information delivery. This is evidenced by the results in the other three components of MES (engagement, emotional connection, and meaningful experience) where though the mean score is higher than the paper-based pamphlet, it is lower than the group using no guide at all. This may imply that the mobile guide design did not best suit the subject matter and the visitors’ needs. This was also reflected in the mean score for the General Usability and Quality Interaction component of the Multimedia Guide Scale (MMGS).

Future works should focus on improving the mobile guide design on a few issues notably on the aesthetics of the visual design component of the mobile guide interface design to evoke emotional arousal similar to the actual surrounding area and utilizing GPS or RFID tags to enable a more accurate mapping between physical and digital environments. It is also important to highlight that the study found that the use of own device has a significant impact on the ease of use as it eliminates issues with learnability and control of the device. Therefore, the BYOD concept should be widely used in informal education institutions implementing mobile guide applications.

In conclusion, the use of MES and MMGS helped us understand the different dimensions of visitors’ experiences with mobile guide technology in public spaces particularly at KOG. Future researches may also explore and improve the developed mobile application that may further boost the visitors’ engagement, emotional connection, and meaningful experience.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

References

T. Ansbacher, “John Dewey’s experience and education: Lessons for museums,” Curator: The Museum Journal, vol. 41, no. 1, pp. 36–50, 1998.
View at: Google Scholar
K. Findlay, Visitor Use of Mobile Devices: The Botanic Garden & Otari-Wilton’s Bush, 2015, https://pdfs.semanticscholar.org/e8f1/5f3118fb33183ec4c0a1b35181feb0c79851.pdf.
M. K. Othman, H. Petrie, and C. Power, “Visitors’ emotions, touristic or spiritual experiences in historic churches: the development of church experience scale (CES),” in Proceedings of the 9th International Conference on Cognitive Science, Procedia - Social and Behavioral Sciences, Elsevier, 2013.
View at: Google Scholar
M. K. Othman, Measuring Visitors’ Experiences with Mobile Guide Technology in Cultural Spaces [Ph.D. thesis], University of York, 2012.
C. A. Templeton, Museum Visitor Engagement Through Resonant, Rich and Interactive Experiences [M.S. thesis], Carnegie Mellon University, 2011, http://repository.cmu.edu/cgi/viewcontent.cgi?article=1024&context=theses.
M. K. Othman, H. Petrie, and C. Power, “Engaging visitors in museums with technology: Scales for the measurement of visitor and multimedia guide experience,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics): Preface, vol. 6949, no. 4, pp. 92–99, 2011.
View at: Publisher Site | Google Scholar
H. Petrie, M. K. Othman, and C. Power, “Smartphone guide technology on cultural spaces: measuring visitors’ experience with an iphone multimedia guide in shakespeare’s church,” International Journal of Human Computer Interaction, 2017, http://dx.doi.org/10.1080/10447318.2017.
View at: Google Scholar
H. T. Zimmerman and S. M. Land, “Integrating Mobile Computers into Informal Science Education,” in Proceedings of the Preparing Informal Science Educators, pp. 169–183, Springer International Publishing, 2017.
View at: Google Scholar
C. Bartneck, A. Masuoka, T. Takahashi, and T. Fukaya, “The learning experience with electronic museum guides,” Psychology of Aesthetics, Creativity, and the Arts, vol. 1, no. 1, pp. 18–25, 2006.
View at: Google Scholar
P. Bell, B. Lewenstein, W. A. Shouse, and M. A. Feder, Learning Science in Informal Environments-People, Places and Pursuits, 2009, http://doi.org/10.1080/00958964.2011.623734.
A. Feix, S. Göbel, and R. Zumack, “DinoHunter: Platform for mobile edutainment applications in museums,” in Technologies for Interactive Digital Storytelling and Entertainment (TIDSE 2004), pp. 3105–2004, Springer-Verlag, Berlin, Germany, 2004.
View at: Google Scholar
E. Klopfer, J. Perry, K. Squire, M.-F. Jan, and C. Steinkuehler, “Mystery at the museum - A collaborative game for museum education,” in Proceedings of the Computer Supported Cooperative Learning (CSCL), International Society of the Learning Sciences, Taipei, Taiwan, 2005.
View at: Google Scholar
N. Proctor and C. Tellis, “The state of the art in museum handhelds in 2003,” in Museum and the Web, Archives and Museums Informatics, Charlotte, NC, USA, 2003.
View at: Google Scholar
F. Sparacino, “The museum wearable?: real-time sensor-driven understanding of visitors interests for personalized visually-augmented museum experiences,” in Proceedings of the Museums and the Web, J. Trant and D. Bearman, Eds., Archives & Museum Informatics, Toronto, Canda, 2002.
View at: Google Scholar
B. Turan and H. Keser, “Museum Guide Mobile App: The Case of the Near,” in Procedia - Social and Behavioral Sciences, vol. 131, pp. 278–285, 2014.
View at: Google Scholar
R. Wakkary, K. Newby, M. Hatala, D. Evernden, and M. Droumeva, “Interactive audio content?: an approach to audio content for a dynamic museum experience through augmented audio reality and adaptive information retrieval,” in Proceedings of the Museums and the Web Conference, 2004.
View at: Google Scholar
K. Walker, “Visitor constructed personalized learning trails,” in Museums and the Web, D. A. Bearman and J. Trant, Eds., Archives Museum Informatics, Toronto, Canada, 2007.
View at: Google Scholar
B. Wagoner and E. Jensen, “Science learning at the zoo?: evaluating childrens developing understanding of animals and their habitats,” Psychology Society, vol. 3, no. 1, pp. 65–76, 2010, http://wrap.warwick.ac.uk/46484/Retrieved from.
View at: Google Scholar
J. H. Falk, E. M. Reinhard, C. L. Vernon, K. Bronnenkant, J. E. Heimlich, and N. L. Deans, “Why zoos & aquariums matter: assessing the impact of a visit to a zoo or aquarium,” Association of Zoos Aquariums, vol. 24, Article ID 0205843, pp. 1–24, 2007.
View at: Google Scholar
J. Falk and M. Storksdieck, “Using the contextual model of learning to understand visitor learning from a science center exhibition,” Science Education, vol. 89, no. 5, 2005, http://doi.org/10.1002/sce.20078.744778.
View at: Google Scholar
R. Malamud, L. Marino, N. Nobis, R. Broglio, and S. O. Lilienfeld, “Do zoos and aquariums promote attitude change in visitors? a critical evaluation of the american zoo and aquarium study,” Society & Animals, vol. 18, no. 2, Article ID 156853010, pp. 126–138, 2010, http://doi.org/10.1163/.
View at: Google Scholar
M. Fleck, M. Frid, T. Kindberg, E. O'Brien-Strain, R. Rajani, and M. Spasojevic, “From informing to remembering: Ubiquitous systems in interactive museums,” IEEE Pervasive Computing, vol. 1, no. 2, pp. 13–21, 2002.
View at: Publisher Site | Google Scholar
N. Proctor and J. Burton, Tate Modern Multimedia Tour Pilots 2002-2003. Mlearn2003: Learning with Mobile Devices, Research and Development, Learning and Skills Development Agency, 2003.
A. J. Pekarik, “The long horizon: The shared values of museum,” Curator: The Museum Journal, vol. 54, no. 1, pp. 75–78, 2011.
View at: Google Scholar
E. J. Brown, Education in The Wild: Contextual And Location-Based Mobile Learning in Action. A Report from The STELLAR Alpine Rendezvous Workshop Series, E. Brown, Ed., University of Nottingham, Learning Sciences Research Institute, Nottingham, UK, 2010, http://oro.open.ac.uk/29882/1/ARV_Education_in_the_wild.pdf.
J. Burston, “Twenty years of MALL project implementation: A meta-analysis of learning outcomes,” ReCALL, vol. 27, no. 1, pp. 4–20, 2015.
View at: Publisher Site | Google Scholar
M. Otter and H. Johnson, “Lost in hyperspace: metrics and mental models,” Interacting with Computers, vol. 13, no. 1, pp. 1–40, 2000.
View at: Publisher Site | Google Scholar
Project Tomorrow, Learning in the 21st century: Mobile devices + social media=personalized learning. Reports from Speak Up, 2012, Retrieved May 16, 2013 from http://www.tomorrow.org/speakup/speakup_reports.html.
Alberta Education, Bring Your Own Device: A Guide for Schools, 2012, http://education.alberta.ca/admin/technology/research.aspx.
D. Rinehart, Students Using Mobile Phones in The Classroom: Can The Phones Increase Content Learning (Proquest Digital Dissertations), 2012, http://gradworks.umi.com/15/17/1517773.html.
Y. Song, “Bring Your Own Device (BYOD)" for seamless science inquiry in a primary school,” Computers & Education, vol. 74, pp. 50–60, 2014.
View at: Publisher Site | Google Scholar
Y. Song and S. C. Kong, “Affordances and constraints of BYOD (Bring Your Own Device) for learning and teaching in higher education: Teachers perspectives,” The Internet and Higher Education, vol. 32, pp. 39–46, 2017.
View at: Google Scholar
J. P. Bowen and S. Filippini-Fantoni, “Personalization and The Web from A Museum Perspective,” in Museums and the Web, D. Bearman and J. Trant, Eds., Archives & Museum Informatics, Toronto, Canada, 2004.
View at: Google Scholar
S. Filippini-Fantoni, J. P. Bowen, and T. Numerico, “Personalization issues for science museum,” in E-learning and Virtual Science Centers, W. H. Leo and S. R. Tan, Eds., Idea Group Publishing, Pennsylvania, PA, USA, 2005.
View at: Google Scholar
S. Filippini-Fantoni, “Personalization through IT in museums,” in Proceedings of the International Cultural Heritage Informatics Meeting (ICHIM '03), Archives & Museum Informatics Europe, Paris, France, 2003.
View at: Google Scholar
A. C. Bertoletti, M. C. Moraes, and A. C. da Rocha Costa, “Providing Personal Assistance in the SAGRES Virtual Museum,” in Museums and the Web, Archives & Museum Informatics, Toronto, Canda, 2001.
View at: Google Scholar
R. V. D. M. Gartner, Gartner Says Tablets Are the Sweet Spot of BYOD Programs, 2014, https://www.gartner.com/newsroom/id/290921.
J. J. Garrett, The Elements of User Experience, Pearson Education, California, Calif, USA, 2nd edition, 2011.
M. Hassenzahl and N. Tractinsky, “User experience research agenda,” Behaviour & Information Technology, vol. 25, no. 2, Article ID 01449290500330331, pp. 91–97, 2006, http://doi.org/10.1080/01449290500330331.
View at: Google Scholar
M. Hassenzahl, “User experience (UX): towards an experiential perspective on product quality,” in Proceedings of the 20th Conference on l'Interaction Homme-Machine (IHM '08), pp. 11–15, ACM, Metz, France, September 2008.
View at: Publisher Site | Google Scholar
M. Hassenzahl, K. Eckoldt, S. Diefenbach, M. Laschke, E. Lenz, and J. Kim, “Designing moments of meaning and pleasure. Experience design and happiness,” International Journal of Design, vol. 7, no. 3, pp. 21–31, 2013.
View at: Google Scholar
J. H. Falk and L. D. Dierking, “The 95 percent solution,” American Scientist, vol. 98, no. 6, pp. 486–493, 2010, http://doi.org/10.1511/2010.87.486.
View at: Publisher Site | Google Scholar
J. Schaeffer and E. Polgreen, Engaging Audiences: Measuring Interactions, Engagement and Conversions, New Mexico, NM, USA, 2012, http://www.j-lab.org/publications/engaging-audiences.
H. L. O'Brien and E. G. Toms, “Examining the generalizability of the User Engagement Scale (UES) in exploratory search,” Information Processing & Management, vol. 49, no. 5, pp. 1092–1107, 2013.
View at: Publisher Site | Google Scholar
J. Pallud, “Impact of interactive technologies on stimulating learning experiences in a museum,” Information and Management, vol. 54, no. 4, pp. 465–478, 2017.
View at: Publisher Site | Google Scholar
M. Lykke and C. Jantzen, “User experience dimensions: A systematic approach to experiential qualities for evaluating information interaction in museums,” in Proceedings of the ACM Conference on Human Information Interaction and Retrieval, CHIIR 2016, pp. 81–90, Association for Computing Machinery, New York, NY, USA, March 2016.
View at: Publisher Site | Google Scholar
L. D. Dierking, J. J. Luke, K. A. Foat, and L. Adelman, “The family and free-choice learning,” Museum News-Washington, vol. 80, no. 6, pp. 38–43, 2001.
View at: Google Scholar
J. H. Falk, “Free-choice environmental learning: framing the discussion,” Environmental Education Research, vol. 11, no. 3, Article ID 13504620500081129, pp. 265–280, 2005, http://doi.org/10.1080/13504620500081129.
View at: Google Scholar
C. W. Glenn, “Cognitive free will learning theory,” Procedia - Social and Behavioral Sciences, vol. 97, pp. 292–298, 2013.
View at: Publisher Site | Google Scholar
A. K. Houseal, C. M. Bourque, K. M. Welsh, and M. Wenger, “Free-Choice Family Learning?: A Literature Review for the National Park Service,” Journal of Interpretation Research, vol. 19, no. 1, pp. 7–29, 2014.
View at: Google Scholar
M. C. Cheh and T. Weng, “Research and development of application of mobile barcode to mobile sightseeing guide on mobile phone,” WSEAS Transactions on Information Science and Applications, vol. 7, no. 1, pp. 16–25, 2010.
View at: Google Scholar
A. Kumar and T. Vithani, “A Comprehensive Mobile Application Development and Testing Lifecycle,” in Proceedings of the IT Professional Conference, pp. 1–27, 2014, http://doi.org/10.1109/ITPRO.2014.7029289.
View at: Google Scholar
L. Naismith and P. Smith, “Context-sensitive information delivery to visitors in a botanic garden,” in Proceedings of the ED-MEDIA World Conference on Educational Multimedia, Hypermedia and Telecommunications, Lugano, Switzerland, 2004.
View at: Google Scholar
L. Naismith, M. Sharples, and J. Ting, “Evaluation of CAERUS: a Context Aware Mobile Guide,” in Proceedings of Mlearn 2005 - Mobile Technology: The Future of Learning in Your Hands, pp. 112–115, Cape Town, South Africa, 2005.
View at: Google Scholar
J. Jeon, G. Chae, and W. S. Yeo, “Developing a location-aware mobile guide system for GLAMs based on TAPIR sound tag: A case study of the Lee Ungno museum,” in International Conference on Human-Computer Interaction, pp. 425–433, Springer, Cham, Switzerland, 2014.
View at: Google Scholar
M. K. Othman, N. E. Young, and S. Aman, “Viewing islamic art museum exhibits on the smartphone: re-examining visitors experiences,” Journal of Cognitive Sciences Human Development, vol. 1, no. 1, pp. 102–118, 2015.
View at: Google Scholar
G. Chen, Y. Zhang, N. S. Chen, and Z. Fan, “Context-aware ubiquitous learning in science museum with ibeacon technology,” in Learning, Design, and Technology, pp. 1–24, Springer International Publishing, 2016.
View at: Google Scholar
L. Neuburger and R. Egger, “An Afternoon at the Museum: Through the Lens of Augmented Reality,” in Information and Communication Technologies in Tourism, R. Schegg and B. Stangl, Eds., Springer, Cham, Switzerland, 2017.
View at: Google Scholar
L. Neuburger and R. Egger, “Augmented reality: providing a different dimension for museum visitors,” in Augmented Reality and Virtual Reality, T. Jung and M. Tom Dieck, Eds., Springer, Cham, Switzerland, 2018.
View at: Google Scholar
E. J. Baker, J. A. A. Bakar, and A. N. Zulkifli, “Elements of museum mobile augmented reality for engaging hearing impaired visitors,” in Proceedings of the 2nd International Conference on Applied Science and Technology, (ICAST '17), vol. 1891, pp. 20–33, AIP Publishing, April 2017.
View at: Publisher Site | Google Scholar
K. Kabassi, “Evaluating websites of museums: State of the art,” Journal of Cultural Heritage, vol. 24, pp. 184–196, 2017.
View at: Publisher Site | Google Scholar
M. Konstantakis, K. Michalakis, J. Aliprantis, E. Kalatha, and G. Caridakis, “Formalising and evaluating Cultural User Experience,” in Proceedings of the 12th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP '17), pp. 90–94, IEEE, July 2017.
View at: Publisher Site | Google Scholar
T. G. Moesgaard, M. Witt, J. Fiss et al., “Implicit and explicit information mediation in a virtual reality museum installation and its effects on retention and learning outcomes,” in Proceedings of the 9th European Conference on Games-Based Learning: ECGBL 2015, R. Munkvold and L. Kolås, Eds., pp. 387–394, Academic Conferences and Publishing International, UK, 2015.
View at: Google Scholar
K. I. Idris, Enhancing Visitors’ Engagement Using Mobile Guide Application at Kuching Orchid Garden (KOG), Universiti Malaysia, Malaysia, 2016, https://www.researchgate.net/publication/304657297_ENHANCING_VISITORS%27_ENGAGEMENT_USING_MOBILE_GUIDE_APPLICATION_AT_KUCHING_ORCHID_GARDEN_KO.
J. Hart, A. G. Sutcliffe, and A. Angeli, Evaluating User Engagement Theory, CHI, 2012.
J. H. Falk and K. L. Gillespie, “Investigating the role of emotion in science center visitor learning,” Visitor Studies, vol. 12, no. 2, pp. 112–132, 2009.
View at: Publisher Site | Google Scholar
J. Chiodo and A. Rupp, “Setting the Stage for Meaningful Exhibits,” in Exhibitionist, The Portico Group, 1999.
View at: Google Scholar
D. G. Oblinger, Learning Spaces, EDUCAUSE, 2006.
N. Bevan, “Measuring usability as quality of use,” Software Quality Journal, vol. 4, no. 2, pp. 115–130, 1995.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2018 Mohd Kamal Othman et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

2847

Downloads

1647

Citations