Abstract
Traditional museums and most digital museums use window display to exhibit their collections. However, the agricultural tools are distinctive for their use value and wisdom contained. Therefore, this paper first proposes a method of virtual interactive display for agricultural tools based on virtual reality technology, which combines static display and dynamic use of agricultural tools vividly showing the agricultural tools. To address the problems of rigid interaction and terrible experience in the process of human-computer interaction, four human-computer interaction technologies are proposed to design and construct a more humanized system including intelligent scenes switching technology, multichannel introduction technology, interactive virtual roaming technology, and task-based interactive technology. The evaluation results demonstrate that the system proposed achieves good performance in fluency, instructiveness, amusement, and practicability. This human-computer interaction system can not only show the wisdom of Chinese traditional agricultural tools to the experiencer all over the world but also put forward a new method of digital museum design.
1. Introduction
China is one of the countries of agricultural origin, and Chinese ancient ancestors create a advanced agricultural culture in this vast land. As the symbol of laboring people’s wisdom, agricultural tools played an important role in the progress of Chinese civilization. Therefore, better reappearance the value of agricultural tools is of great significance [1].
Most traditional museums use window display to show their collections, which is rigid and boring. In recent years, the emergence of digital systems provides a new way to present with the development of virtual reality, 3D model and other technologies [2, 3]. The immersive sensibility of VR technology has multiperception, which can simulate human touch, vision, hearing and motion perception, so it can bring more realistic game experience to users. The application of VR technology in museum display will break the time and space constraints of traditional museums [4]. Meanwhile, Unity3D is one of the most famous virtual reality tools, which is the development platform [5]. Because of cross-platform development, it is often used as a development platform for virtual scenes. The platform uses 3D models for effective construction to truly restore real-life scenarios.
The researchers have started to use it to preserve and demonstrate cultural resources. Xiang Hui et al. designed and implemented the archaeological digital museum of Shandong University based on the virtual reality technology [6]; Peng Guobin et al. analyzed the characteristics of virtual design and resource objects, and accomplished the digital exhibition of Dongjiang Ecology Museum in Sanjiang [7]; Selma Rizvic et al. fulfil the Interactive Sarajevo City Model application with a set of 60 photographs taken around the physical model as central ObjectVR and 18 points of interest, all over the city [8, 9]. In addition, “Forbidden City beyond time and space” uses the panoramic images and virtual reality technology to display the digitized “Virtual Forbidden City” to the world [10, 11]; the Taiwan’s National Palace Museum (NPM) redesigned the online 3D Virtual Artifacts Exhibition System, bringing experiencers utterly new experiences on artifacts [12] and so on. Digital systems mentioned above have realized the virtual display of ancient Chinese culture, which has enabled Chinese culture to transcend the geographical boundaries and head to the world. However, there are still some problems in these systems, on one hand, the panorama of the builds and scenes in many digital display systems causes of limited scenes space and lacking of authenticity [13], especially when the picture is stitched or different equipment is used; On the other hand, user experience is not good for lacking of the interaction between the experiencers and the system. Although the number of exhibition articles is large, the information transmitted is little, leading to the values contained in the exhibits have been neglected.
In traditional Chinese culture, agricultural tools embody the wisdom in ingenious design and the value in use, which are different from porcelains, sculptures and other items. Therefore, more interactions are required between agricultural tools and experiencers [14]. However, the experience of agricultural tools in the real fields also be restricted by environment, season and safety. Especially in big cities, the education of traditional agricultural tools is very difficult. This paper proposes a virtual human-computer interaction system by building a digital agricultural tools museum with VR equipment [15, 16]. Different from the traditional museum, a method of multichannel interactive exhibition is put forward for the introduction of agricultural tools. Instead of the single view in the roaming process, an interactive roaming method is proposed. Meanwhile, in order to vividly display the dynamic using process of agricultural tools, this paper arranges interactive activities in virtual outdoor farming scenes, illuminated by the game design ideas. The proposed system enhances the immersion of experiencers and solves the problems concerning environment, season, and security, which is of great significance to the development of China’s agricultural culture education.
2. Main Ideas
The system platform mainly contains the following three aspects: building models for elements in traditional agriculture, including building models, agricultural building models, crops models, animal models, and natural landscape architecture models; build scenes of the indoor agricultural tools museum and the typical outdoor farming; the design of virtual human-computer interaction tasks. The framework of the whole system is shown in Figure 1.
This paper begins with the investigation and collection, and extensive data provides reliable support for subsequent modeling of tools and scenes. Three-dimensional model rather than panorama is used to get the real effect. Consequently, farming scenes which consist of three-dimensional models highly reproduce the effect of outdoor farming.
In order to provide a comprehensive introduction of agricultural tools, the scenes consist of two parts: indoor agricultural tools museum and the outdoor farming. Indoor agricultural tools museum mainly uses exhibition windows, static displaying the agricultural tools. Meanwhile, experiencers can dynamically interact with the agricultural tools through the mouse and keyboard control during virtually roaming. Furthermore, experiencers can dynamically operate the agricultural tools in outdoor farming scenarios. In outdoor farming scenes, the experiencers can accomplish the tasks of agricultural farming activities with VR equipment, dynamically learning the use and value of agricultural tools. The combination of the static display and the dynamic operations is helpful in understanding agricultural tools’ knowledge on cultural historical period, origin, production process, production principles, and so on.
3. Establishment of 3D Model
In this section, 3DsMAX [17] technique is used to reduce the physical characteristics of agricultural tools models. Besides, 3D models will be built in strict accordance with the real specifications. On the other hand, the simplex principle is adopted in the following models: crops models, animal models, building models, and natural landscape architecture models.
The establishment of the model is the foundation of the virtual scenes. The main methods of the modeling are as following: firstly, Nonuniform Rational B-Splines (NURBS) [18] surface modeling and polygon modeling are applied in the modeling tools. Secondly, the model editor grid can ensure the details of the models. Then, to achieve the desired shape effect of the models, the high simulation models are created by bending, rotating, angle capturing, mirroring, and super Boolean operations on the models. Finally, UVW mapping [19] is used to increase the authenticity of the models. What is more, the appropriate parameters for the bump mapping diffuse reflection and highlight will make models more realistic. The process of the keel waterwheel model is shown in Figure 2.
(a) Grid editing
(b) Angle capture
(c) The module stitching of additional line
(d) The render effect of module
In addition, for the fluency, perfect performance and low calculation consumption [20] of the system, patch modeling and hierarchical modeling are used in the modeling process of natural landscape models; but, there exists the simplicity structure problem of plants derived from the above ways. To solve this question, the texture of the plants photographed in the real life is used in the models. Finally, models and materials are exported, saved as FBX format. Abstract phenomena such as rain, snow, fire, water currents, and other unnatural landscapes are modeled using the particle system in Unity 3D [21].
4. Construction of Virtual Scenes
4.1. Scenes Construction of Indoor Agricultural Tools Museum and Outdoor Farming
According to the different roles of the indoor and the outdoor to build different scenes in the layout of the scenes, the indoor exhibition scenes aim at displaying the tools, while the outdoor farming scenes aim at restoring real farming scenes. Further, according to the difference between the northern tools and the southern tools, the second division of different scenarios is built. Finally, the further division is made based on the different cultural elements. The hierarchy of the scenes is shown in in Figure 3.
The agricultural culture contains a great deal of farming elements including types of agricultural tools in north, various agricultural tools in south, typical agricultural books, and historical figures and various agricultural pictures. Therefore, this paper presents an agricultural information classification method that can array the wide variety of cultural elements in a reasonable way. This method not only can guarantee as comprehensive as possible to summarize the traditional agricultural characteristics but also can sort out the various types of agricultural cultural elements in different levels. The flowchart of agricultural tools classification that display in the museum is shown in Figure 4.
According to the difference between the south and the north, the agricultural cultural elements are classified as follows: if the cultural elements of the agricultural tools are the northern, they will be displayed in the northern exhibition hall. Otherwise, they will be placed in the southern. To judge whether the agricultural cultural element is the agricultural tool, if so, the animation demonstration will be added to it. Furthermore, targeted at typical agricultural tools which need to be highlighted, an interactive control demonstration of these tools will be added [20], and it will also be added to the outdoor farming experience interaction tasks. Otherwise, go to the next step. Whether it is a typical agricultural book or a historical figure that has outstanding contribution to the development of agriculture, if so, it will be displayed in the static window adding introduction to it. If is not, go to the next step. To determine whether it is a picture display relevant to the agriculture, if so, hang the pictures for static display and add the corresponding audio and text introduction to it. Thus, the agricultural introduction will be clear and systematic.
In this section, Unity3D is used as the development software. The 3D models are exported, in the form of.FBX, and then poured into the Unity3D platform to build the scenes. According to the different characteristics of the imported models, the models can be roughly divided into three parts: static model, models with frame animation, and the models of natural landscape.
There may be some issues arising along with the model import. First, when the models are imported into Unity3D, two means are taken to avoid material losing. One is to package the materials and models of each model and then import the whole package into Unity3d. Another is to combine multiple maps of the same model into one to prevent the texture maps losing. Second, aiming at animation model, a poor effect may occur after importing. Thus, it is necessary to reorganize the action relationship of animation [22, 23] and adjust the animation to a reasonable setting. Third, as for problems of transparency when the models of facet structure are imported, for example, natural landscape models, it is essential to reset the parameters of the facet structure models to achieve a better effect. The display effects of virtual scenes are shown in Figure 5.
(a) The scene of southern exhibition hall
(b) The scene of northern exhibition hall
(c) The display of farming tool
(d) The display of farming books and figures
(e) The scene of northern farming
(f) The scene of mulberry fish ponds
The combination of indoor exhibition scenes and outdoor farming scenes gives experiencers comprehensive understanding of farming culture elements in different perspective. It is user-friendly details of the tools for viewing in the indoor museum. Meanwhile, the outdoor farming scenes make up the deficiency of indoor scenes. It is useful that operating tools in outdoor specific agricultural activity during the agricultural season.
4.2. Optimization of Scenes
In order to ensure that the system has universal adaptability, the virtual scenes in the system will be optimized from the following three aspects:
The authenticity of scenes can be improved by using higher precision models, but such fluency is terrible. LOD technology [24] can be used to make lower precision models, which can ignore the unimportant details. The alternate use of higher precision models and lower precision models can reduce the rendering consumption achieving a good performance in fluency during running. The system may then decide whether to take action to alternate model as a result of the spatial distance between targeted model and the main camera. If the distance is larger than the setting threshold, the higher precision models will be dynamically replaced with the lower. The flowchart of camera and model distance detection algorithm is shown in Figure 6.
In this section, to accelerate the loading speed of the models, the plug-in named Simple Mesh Combine is installed which can combine the similar models together. It can effectively reduce the number of draw calls; thus the consumption on the CPU side decreases. At the same time, a multi-to-one method of merging multiple maps into one shared in multiple models can also effectively reduce the amount of materials to improve the loading speed.
Since the authenticity of the scenes is crucial for experiencers, there is no delay to light the scenes. A number of lights are added in order to get the authenticity in different scenes. Meanwhile, the fluency is terrible. For the authenticity and the fluency in scenes, the technology of baking map is used in this section. When modeling, especially for the effects of museum’s lighting and the shadow, firstly adjust the corresponding parameters to get a real effect; secondly use the 3Ds MAX for baking [25], which converts the light information into textures; then apply the baking textures to the scenes. Finally, after the model’s FBX are introduced into scenes, few lights will achieve perfect effects.
5. Design and Implementation of Human-Computer Interaction Technology
To avoid the deficiencies existing in the traditional museums, a human-computer interaction system is proposed in this system. This system can include not only information associated directly with tools but also interaction that is necessary for presenting the tools to experiencers. It is particularly important that interactive systems are operated in introduction, when experiencers can utilize the system not only by browsing objects but also by interacting with the tools.
In this section, four human-computer interaction technologies are adopted to design and construct a more humanized system as follows: intelligent scenes switching technology; multichannel interactive introduction technology; interactive virtual roaming technology; task-based interactive technology.
5.1. Intelligent Scenes Switching Technology
In this section, two different switching modes are proposed according to different scenario: list switching method; triggered switching method.
(1) List Switching Method. The list switching method is adopted, when the experiencers enter into indoor museum scenes. This method offers limited choices for the experiencers, so it can help experiencers enter into the target exhibition hall quickly. Additionally, to prevent experiencers staying one position too long, a time threshold is set. If the experiencers do not make a corresponding selection in setting time, the system will automatically enter the default scene. This method reduces timing overhead.
(2) Triggered Switching Method. In indoor museum scenes, there exist a large number of agricultural elements, which are placed in in different places. When faced with so many choices, the list switching method increases timing cost. Compared with the list switching method, the triggered switching method is adopted. Experiencers can get the knowledge of the model in front of their eyes timely, when walking around a farming tool. This method works well on a widely variety of requirements-specific handoffs and switches specifically the introduced scene of the target agricultural tools. The flowchart of scenes triggered switching algorithm is shown in Figure 7. The triggered switching effect of tools introduction is shown in Figure 8.
(a) Add collision body
(b) The mouse enters the range of collision body
Add a collision body for each tool model and set an appropriate collision body range to it.
Use the EventTrigger, a component of Unity3d, to monitor the variable of “OnMouseEnter” to see whether the mouse has entered the setting detection range. If true, then text prompts “Click to view more information about xx.”
Use the EventTrigger to monitor the variable of “OnMouseDown” to see whether the experiencers click the mouse. If true, then switch to the outdoor farming scenes with a specific introduction.
Using intelligent scene switching technology, experiencers can focus on interested tools, effectively reducing the disorder of scenes switching and time consuming during switching.
5.2. Multichannel Interactive Introduction Technology
Traditional museums introduce exhibit by audios and pictures. In this section, multichannel interactive introduction technology is proposed to overcome the deficiencies in traditional museums. Compared to the tradition museum, this technology assembles the video, audio, text and it can mobilize the sensory organs such as visual sense, acoustic sense, and tactile sense [26] making the introduction more diversified. The framework of multichannel interactive introduction is shown in Figure 9.
Interactive control of typical agricultural tools models mainly contains following aspects. At first, experiencers can trigger the instruction by the input module of the keyboard or mouse; then the background channel passes the integrated messages to the interactive control system, in which the instruction information can be handled; finally, the feedback instructions will be separately applied to the external display device transmitting to the experiencers. Additionally, experiencers can interact with the 3d models by using hand motion (zoom in, zoom out and rotate) which increases the interaction with the system as in [21]. The effect of multichannel interactive introduction is shown in Figure 10.
5.3. Interactive Virtual Roaming Technology
Virtual roaming is a real-time browsing of virtual scenes [27], which not only utilizes automatic scene sightseeing, but also realizes human-computer interaction in three-dimensional scenes.
In this section, to satisfy different requirements of experiencers in different virtual scenes, two different roaming methods are introduced: interactive iTweenPath virtual roaming method progressive virtual roaming method
(1) Interactive iTweenPath Virtual Roaming Method. The iTweenPath roaming method uses a number of key points provided by the iTween plug-in to generate the roaming route. Nevertheless, this method still has some problems which are not in conformity with the real-life human behavior pattern, resulting in poor experience. One problem is the single point of view. The viewing angle of the experiencers is always in one direction, while the roaming position changes. In [21], the method of tangent is adopted to solve this problem, but the interaction is poor. Another problem is that it can’t be suspend in the middle. It is inconvenient for experiencers to sightseeing.
In view of the above problems, interactive iTweenPath virtual roaming method is proposed in this section. In this method, the pause and restart keys are added. During roaming, the experiencers can pause at any time and observe any part they want to see. What’s more, a mouse rotation function is added. When the experiencers suspend during roaming, they can view in all directions through the mouse control. The interactive virtual roaming algorithm of iTweenPath is shown in Figure 11. The camera’s direction of the original iTweenPath roaming path VS the camera’s direction of the interactive iTweenPath roaming path is shown in Figure 12.
(a) Original iTweenPath
(b) Interactive iTweenPath
The specific process of the algorithm is shown following:
The algorithm script is mounted under the camera. When it starts to run, record the initial position of the camera as place_old and the initial direction as Cdir_old;
If the camera has not completed the path roaming, enter to the following steps, otherwise it is deemed that the roaming has been completed and the camera stops moving;
Use EventTrigger to detect keyboard space input. If running before, then roaming pause and voice introduction pause and record the location of the camera in the current path as place_new and the time of voice introduction as music_time; if the previous is pause state, then reenter the roaming state. The records will be updated: place_old = place_new, Cdir_old = Cdir_new;
Use EventTrigger to detect whether the mouse moves or not, if the angle of the camera is changed and record the current angle as Cdir_new.
The method of interactive iTweenPath virtual roaming not only solves single point invariant problem of the iTweenPath in the roaming process, allowing users to observe in diversified directions during the roaming process, but also increase the human-computer interaction overcoming the weak interaction in the roaming process [21]. According to their own interest, experiencers can visit the indoor museum in an all-round way. It is more suitable for people's viewing habits in life.
(2) Progressive Virtual Roaming Method. In this section, while achieving the movement from one endpoint to another, progressive virtual roaming method is proposed to maximum cut the consumption of path loading. It is based on the C# script, in which both the starting point and the ending point of the roaming are added. Compared with the interaction iTweenPath virtual roaming method, it is suitable for a short distance straight roaming situation. For example, at the initial stage of entering the agricultural tools museum, experiencers achieve the effect of gradual promotion of the lens when entering the museum.
5.4. Task-Based Interactive Technology
In this section, a task-based interactive technology that draws inspiration from the game is developed. Meanwhile, with the help of VR display device and VR control device, this system realizes the interactive experience of tasks. In order to restore the agricultural tools and the typical farming scenes, they are divided into five parts: the interaction of wheat harvesting and processing; the interaction of corn planting and processing; the interaction of cotton picking and weaving; the interaction of rice planting; ecological cycle interaction of mulberry fish pond.
5.4.1. The Convergence of Tasks
Farming activities are reflected in the alternate use of multiple agricultural tool. Single agricultural tool is not enough to demonstrate their characteristics. Thus, the task-based interactive technology divides these activities into a series of smaller tasks. Then, these smaller tasks are connected in the chronological order of crops growth. This technology enables experiencers to have a deeper understanding of the crops growth process and agricultural activities. In this section, the bidirectional linked list is adopted to fulfill the tasks orderly. The task list is shown in Figure 13. At the same time, a time threshold is set to solve time stagnation and other causes leading to time overhead. If the task is not completed within the setting time, the task will be suspended and then enters into the previous task.
5.4.2. The Implementation of Tasks
Taking wheat harvesting and processing activity as an example, it is partitioned into seven smaller tasks, and it embodies the coherence of the farming process. The flowchart of wheat task is shown in Figure 14.
The process of wheat harvesting and processing mainly includes the following tasks: (a) pick up the sickle and walk to the wheat field; (b) harvest wheat; (c) place the harvested wheat under stone roller; (d) catch cattle and roll the wheat; (e) pick up the rolled wheat to the windmill; (f) pour the wheat mixed with debris into the Windmill and turn Windmill; (g) finally, get clean wheat grains. The effect of wheat virtual human-computer interaction is shown in Figure 15.
6. Experience Results and Evaluation
6.1. System Internal Parameter Evaluation
The operating environment of this system is based on graphics workstation with CPU 2.2 GHz, 32GB of memory, and hard disk 1T. The used VR equipment is HTC Vive external head display [28]. In the PC-side, the frame rate FPS comparison of the scene operation before and after is shown in Table 1.
During the runtime, the larger the FPS is, the smoother the system will be. In general, 60 frames are very smooth at runtime. After using optimization method mentioned in the Section 4.2, the number of frames in the indoor museum scene has been greatly improved, which can be seen from Table 1. The average frame rate reaches 110.2 frames per second while the average frames rate is 10.3 frames per second before optimized, which increases approximately 10 times. In outdoor farming scenes, the average frames rate reached 79.2 frames per second after optimization, while the average frames rate is 38.5 frames per second before optimization, which increased approximately 2 times. The frame rate in both scenes is higher than 60 frames per second, which shows that the virtual scenes are running well.
6.2. System Experience Evaluation
In this section, the proposed system and the China Agricultural Digital Museum are compared mainly from the objective indicators and user experience.
6.2.1. Objective Indicator
The two digital systems are mainly compared from the following aspects: the form of the model, the construction of the scene, the viewing angle of the user, the user viewing path setting, and introduction methods of agricultural tools and agricultural books. The specific settings are shown in Table 2.
The China Agricultural Digital Museum is characterized by computer web browsing, so there are low requirements for equipment. Different from above, the proposed system makes good use of computer platform and VR devices for better user experience; the construction of scenes in the China Agricultural Digital Museum are mainly in the form of panoramic maps, which causes scene deformation shown in Figure 16. At the same time, agricultural tools, agricultural books, and other parts are mainly introduced by static pictures and texts. In the proposed system, the construction of agricultural tools model and the scene environment are based on the simulation of 3d models, which solves the deformation of the panorama during the viewing process and rebuilds agricultural tools and agricultural scenes at maximum. What is more, various introduction methods, like as sounds, videos, and texts, are combined for the presentation of agricultural tools, and the user even can interact with the system through the VR device.
6.2.2. User Experience
In addition to making a better evaluation of the system, “hands-on” operation and a questionnaire were provided to ten students to comment on five aspects of the system including authenticity, fluency, instructiveness, amusement, and practicability. The evaluation results are shown in Table 3. When it comes to the authenticity of the scenes, nine students think that the scenes of the system real rebuild the typical farming scenes. In regard to the fluency of the system, four students feel it was good, and six considered that it was better, and the result shows that the system is running fluently. As to the instructiveness, two students feel it is good, and eight considered that it is better. In terms of the amusement, all of the ten students think that the interactive process of agricultural activity is more interesting than the traditional education. Relating to the practicability, there is only one student who feels bad, four think good, and five think better, and 90% students assert the system is practicability. By analyzing evaluation, some students unfamiliar with VR devices and rules about the interaction in the system result in the experience of the system being bad in operation.
In a word, the system achieves good performance in five aspects: authenticity, fluency, instructiveness, amusement and practicability. The experiencers have provided very positive feedback and shown greater excitement at the applications of this system than the original digital museums.
7. Conclusion
In this paper, a human-computer interactive system for agricultural tools museum is described. In comparison with the existing digital museum, this proposed system increases the visual immersion and interests of experiencers. Moreover, it provides a new display way for the development of digital museum. This paper proposes three aspects conclusions in digital museum. They are listed as follows:
In this paper, a new display method of agricultural exhibit is proposed. This proposed system introduces the background of the various agricultural tools, the principles, the evaluation process, and so on in detail, which can cope with the unicity problem when introducing traditional agricultural tools. Experiencers can interact with the agricultural tools with VR equipment. At the same time, inspired by the game, the interactive farming activities are arranged in the chronological order of crops growth in human-computer interactive process.
In the construction of virtual scenes, a classification way of various agricultural elements is presented. It can classify the elements organically and hierarchically. Clearly, the diversity of agricultural tools culture elements can be reasonably sorted out.
For human-computer interaction, a humanized human-computer interaction method is proposed. Intelligent scene switching method can make the switching of scenes more humanized; the introduced way of multichannel overcomes the unicity shortage and enriches the introductions of agricultural tools; Further, interactive virtual roaming method provides experiencers with a comfortable viewing experience. Additional, task-based interactive technology lets experiencers receive education in entertainment. The proposed methods above not only overcome problems existing in the previous virtual interaction but also increase the comfort of the experiencers during operation.
The evaluation proves that compared with the China Agricultural Digital Museum, the system has better effect on authenticity, fluency, instructiveness, amusement, and practicability. Nevertheless, in terms of practicability, there are certain requirements for the operating space owing to the connection of external VR devices. Further investigations are underway to improve the fluency and practicability of system operation and optimize the system to obtain a better experience. In the future, we also can create an educational platform and make VR application available and ready to be used in different subjects to support learning process.
Data Availability
The results in this article are entirely theoretical and analytical. The main steps of the demonstrations for each results are clearly reported in the text and the article is fully consistent without the support of any additional data.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported in part by Research on Cultural Resources Modeling Technology and Digitization of Characteristic Cultural Resources (2015BAK04B01), which belonged to Digital Technology Application and Demonstration of Grassroots Public Cultural Services Program (2015BAK04B00). This program was organized by Film and Television of the People’s Republic of China.