Abstract
As a form of animation that is closest to the essence of movies among all kinds of animation, stop-motion animation is deeply loved by people and has a lot of room for development in the new era. The steady progress of science and technology has raised people’s requirements for the presentation form of stop-motion animation. People are no longer satisfied with the design of stop-motion animation, which only unilaterally transmits information to the audience and does not interact with the audience. To meet people’s needs, this paper applied the augmented reality technology of digital media to the design and production of live-action stop-motion animation. It used 3D tracking and registration technology to integrate virtual characters into real-life stop-motion animation, and presented it to the audience through augmented reality equipment to improve the interaction between real-life stop-motion animation and the audience. Finally, this paper conducted an experimental design on the fluency, accuracy, animation playback, and user satisfaction of the real-life stop-motion animation of augmented reality. The results showed that the smoothness and accuracy of the live-action stop-motion animation using the augmented reality technology of digital media have been correspondingly improved, and the user’s satisfaction with the augmented reality live-action stop-motion animation has increased by 3.1%. The augmented reality technology of digital media has greatly improved the interaction between live-action stop-motion animation and the audience and promoted the development of live-action stop-motion animation.
1. Introduction
Stop-motion animation often adopts the principle of “persistence of vision”, by shooting real objects one by one and then making them continuously show, so as to achieve the illusion that the things that are photographed are swaying. Animation is a language of expressing art, and the appearance of language is to pave the way for communication. Because of this desire to communicate, interactive technologies are developing. Many fields are exploring ways to enrich information transfer and interaction, and the field of animation is no exception. Most notably, augmented reality for digital media is constantly being developed and used for stop-motion animation. Augmented reality animations allow for ever-improving interactive animations, enabling the use of real-life situations in animations. The application of augmented reality technology to the production process of stop-motion animation can make people feel immersive when watching cartoons, and also allow people to interact with the characters in the animation, which plays an important role in promoting the development of stop-motion animation.
To promote the development of live-action stop-motion animation, different scholars have invested in the design and production of live-action stop-motion animation. To allow live-action stop-motion animation to capture the effect of moving objects on film with limited time exposure, G Brostow used image tracking technology to track the frame-to-frame motion of objects in the image plane to the extent of interactive control-induced blur [1]. H W Wang expounded on the four characteristics of stop-motion animation and the four differences between the action design in stop-motion animation and the most used drawing materials in animation [2]. To prevent stop-motion animation from affecting the middle and postproduction process and final effect of the film, Y Sun summed up four ways of thinking for creation by experimenting with the creation of live-action stop-motion animation:: material collection, construction mode, design coordination, and audio-visual communication [3]. To expand the material application of stop-motion animation, R Xiao in the teaching practice process of animation classroom film, let students master the material form and production skills to make stop-motion animation [4]. Liu, Xin completed a reasonable planning and production process to study the application of commercial design and production methods in personal stop-motion animation. He documented the creative process and content production of the film in detail, from the creation of the project concept to the actual production, which promoted the development of stop-motion animation [5]. AK Nassi explored the production process of stop-motion animation through an artistic research method, discovering that to create a successful work, one must work together and must pay attention to every detail [6]. Cut-out animation is one of the most ancient animation techniques. Purwaningsih et al. [7] finally designed a paper cat puppet that was widely used by studying existing paper puppets in other kirigami shorts, tutorials, commercials, and behind-the-scenes videos of filmmakers [7]. These studies show the continuous development of live-action stop-motion animation. Although these studies have achieved rich results, with the emergence of new technologies and new situations, stop-motion animation has a problem of low interactivity, which requires more attention to the interactivity of live-action stop-motion animation.
The popularization of augmented reality technology of data media makes more and more researchers invest in augmented reality research. AH Behzadan investigated the application of outdoor augmented reality in 3D graphics simulation of construction activities and found that augmented reality can be used with corresponding peripheral devices to generate mixed views of the real world and superimposed virtual building graphics [8]. J Aleotti proposed an approach to visually augmented reality and creates an interactive environment. He has also developed precise algorithms for camera calibration, registration, object manipulation, and feedback computation [9]. To improve the learning experience and improve academic performance, H Permana applied augmented reality to the course to visualize the phenomena obtained from the laboratory research results and improve course interactivity [10]. Faridi et al. [11] displayed chemical equations in the form of augmented reality animations, attracting the attention of scholars and deepening their understanding of the concept of equations, improving the interaction with equations [11]. To speed up the restoration of historical sites, AAG applied augmented reality technology to the restoration of historical sites, displaying virtual images of historical sites and speeding up the restoration of sites [12]. Klaus Chmelina explained and defines terms such as virtual reality, mixed reality, and augmented reality. He introduced and discussed two different application scenarios of geotechnical engineering, where virtual tunnel inspection can be performed [13]. N Pantuwong proposed to use augmented reality to reduce the difficulty of computer animation creation, a new interactive technology controlled by 3D character animation. The user does not need to interact with the typical mouse and keyboard to control the animation, but only with tangible objects attached with a few visible markers [14]. The above research showed that augmented reality technology can improve the interaction between live-action stop-motion animation and users.
As digital devices have advanced, hardware has evolved from computers to portable digital media devices. Some digital media devices have built-in applications related to augmented reality technology, which can be used as a platform for displaying augmented reality animations [15, 16]. The development of stop-motion animation is inseparable from the application of technology. And digital media augmented reality technology can make the development of stop-motion animation more extensive. In this paper, the camera is used to take real photos, and the augmented reality technology is used to integrate the three-dimensional virtual animation image into the real scene and transmit it to the audience through the augmented reality device.
The innovations of this paper are as follows:(1)This paper studies the combination of augmentation technology and stop-motion animation and improves the theoretical research of augmented reality animation.(2)In this paper, the fluency of augmented reality stop-motion animation is higher than that of traditional live-action stop-motion animation, which can bring a better look and feel to the audience. The accuracy rate of augmented reality stop-motion animation is higher than that of traditional live-action stop-motion animation, and the error rate is smaller in production.(3)This paper summarizes the differences between augmented reality stop-motion animation and traditional live-action stop-motion animation and improves the interaction between live-action stop-motion animation and the audience.
2. Stop Motion Animation Method
With the development of the Internet, stop-motion animation is facing unprecedented challenges, which puts forward higher requirements for the novelty, creativity, and material sharing of stop-motion animation. The stop-motion animation production of digital media is based on the use of digital media and uses augmented reality technology as a means to achieve the purpose of combining stop-motion animation with real scenes. There is no digital programming without digital equipment, so the automation and networking of the stop-motion animation production process is an indispensable prerequisite for digital programming.
2.1. Stop Motion Animation Production Network Platform
The system covers the whole process of animation production and should follow the principles of security, structure, and openness. Each module must maintain good synergy while ensuring independence, not only to work perfectly during operation but also to work independently as an independent system to ensure the security of the entire network [17]. As shown in Figure 1, the network platform includes four modules, a stop-motion image acquisition module, an audio information acquisition module, a stop-motion animation output module, a user module, and the main control module. The platform uses a camera in the stop-motion image acquisition module to shoot real-life materials and stop-motion animation materials, and then put them into the admission unit through a computer. After the audio information acquisition module uses the recorder device to record the audio required for the stop-motion animation, it is transmitted to the audience through augmented reality glasses. After the audio information acquisition module uses the recorder device to record the audio required for the stop-motion animation, it is recorded in the main control module. It uses augmented reality technology to combine virtual and real, and transmits it to the audience through augmented reality glasses.
2.2. Real-Life Stop-Motion Animation Reality Equipment
The actual display device for live-action stop-motion animation mainly refers to augmented reality helmets. There are two main types of augmented reality helmets: optically transparent helmets and transparent video helmets [18]. Of course, according to the number of projectors, it can also be divided into the monocular head screen and binocular head screen. This article uses an optically transparent helmet. For the optically transparent helmet, an augmented reality camera needs to be added, and the positional relationship between the camera and the helmet screen needs to be calibrated. This article uses an optically transparent helmet. For the optically transparent helmet, an augmented reality camera needs to be added, and the positional relationship between the camera and the helmet screen needs to be calibrated. The most common method is the single-point active alignment method, which requires the user to wear a head-mounted screen and align some cursors on the screen with the real object multiple times, which is done by rotating the head. It is necessary to rotate the head to align with the cursor to see the real object. As shown in Figure 2, the DLT method is used to construct a system of equations to solve the projection panel.
2.3. Human-Computer Interaction Design
The definition of human–computer interaction is divided into broad and narrow senses, and the generalized human–machine interface is a key component of the human–machine system. The human–machine system consists of three parts: human, machine, and environment, which are interconnected as a whole. There is an interactive interface between human and machine, which is called human–machine interface, that is, the information reception and exchange between human–machine is based on the human–machine interface [19]. People receive information through screens and make appropriate judgments and decisions in their brains, then control controls, enter commands to be executed, and machines receive and execute commands. The corresponding programs are displayed and disseminated in this way, and the human–machine interface bears the main link of the human–machine information transmission in the entire circulatory system. It turns out that the design of the human–machine interface has a great relationship with the functionality of the human–machine system and the rationalization of the human–machine interaction, as shown in Figure 3.
The human–machine interface in the narrow sense refers to the human–machine interface (HCI) in the modern computing environment, which is called the user interface (UI). It is the carrier and means of receiving and exchanging information between people and computers, and it is the computer system that provides a complete operating environment for system users. Human–machine system is a human–machine system composed of people, computer hardware, and computer software, and the human–machine interface is the intersection of the three parts. The working process of the man-machine system is as follows: the human–machine interface transmits information to the user, the user receives the interface information, and uses the interactive device to complete the human–computer interaction. The computer receives user information commands and runs appropriate programs and provides information about the projector to the user. Among them, computer technology has realized the transformation from hardware to software, and the relationship between human–machine interface and human–machine interface is handled and solved by interface designers. To sum up, the man-machine interface is a means of transmitting and exchanging information between the user and the computer, and between the human and the computer, which provides a complete user interface for the user to use the computer. The development of human–computer interaction gave birth to the human–computer interface. The human–computer interface has opened its historical chapter since the birth of human–computer interaction.
2.4. Network Architecture
Due to the characteristics of animation applications, the key elements of the network architecture must be high performance, large capacity, high reliability, and easy scalability. The key element of the network architecture is in the embedded network platform. The front-end processing application system is composed of high-speed Gigabit Ethernet, and the back-end server storage system adopts the technical storage area network, which utilizes the storage resource allocation and management provided by virtualization. The storage function saves the cost of managing multi-channel storage devices while performing collection, production storage, and business expansion in unlimited geographic space. The advantages of remote storage are far greater than the disadvantages of complex operation and management and limited connection distance. In the environment where Gigabit Ethernet is popular, the characteristics of easy expansion, high performance, and high availability are also important reasons for choosing this solution. The network architecture is shown in Figure 4.
3. Recommendation Algorithm
Augmented reality systems need to use cameras to locate and orient the phone’s real scene, so that real-time information can be used to determine the correct location of the 3D virtual model to be added to the real world to complete the registration augmented reality system lock.
3.1. Nonlinear Space Expansion of Stop-Motion Animation Images
A stop-motion animation image is a spatial illusion created by various artistic means in a flat space. This spatial illusion is the visual state of an object at a specific time and place, which is a static level. The spatial expansion of visual images in the digital age aims to break the shackles of plane space and expand the static spatial form of traditional optical images into a three-dimensional motion form [20]. A linear path is the transmission of information in one direction, which the linear expression does not change, and the sender and receiver are separated, as shown in Figure 5.
Digital media has changed the simplification of image dissemination and formed dissemination networks, breaking the traditional point-to-point linear way of visual images, each of which can transmit image information to each node of the grid in a nonlinear way. Nonlinear features allow visual images to be arbitrarily cut and then reassembled, giving the viewer a sense of temporal and spatial chaos, as shown in Figure 6.
3.2. Camera Calibration and 3D Tracking Registration
Since the 2D augmented reality registration always assumes that the coordinate value of the sign space is zero, therefore, for the 3D augmented reality registration, the matrix A cannot be applied to the 3D augmented virtual target object in the image space alone. The lost camera parameters can be recovered by using the plane correspondence and camera calibration to complete the 3D registration. equation (1) cannot be applied only to 3D augmented virtual objects in image space. The plane matching and camera calibration data should be obtained, completing the capture of 3D data for missing camera settings [21, 22].
B is a rotation matrix and satisfies the following relationship:
Similarly, there are:
Eliminate of (3) and (4) to get:
3.3. Derivation and Improvement of 3D Registration Algorithm Based on Logo
The three-dimensional registration based on a logo is one of the visual registration methods, so the amount of image processing is large. In this paper, the coplanar black and white two-color cardboard logo is placed on the wall or desktop, and the real-time video of the logo is captured by a camera with a small radial distortion installed on the helmet display. The video image sequence captured by the camera is input into the computer (PC) through the USB interface, and then the 3D-2D feature point matching method is used to determine the user’s viewpoint orientation to realize the three-dimensional registration of the system.
A perspective view is generally used to project a 3D virtual model onto a computer screen, but for a lens with slight radial distortion, the relationship between any point in space and its displacement in the image can also be viewed as a calibrated and accurate perspective view. The internal and external parameters of the lens can be completely superimposed on the simulation model and video image of the same object [23]. There is the following variation relationship between any point P (M, N, Z) in the world coordinate system space and its corresponding point on the computer screen:
Among which,
3.4. Correction of the Image
To match the preestablished template with the picture to determine the ID of the marker, the picture must be geometrically transformed to make the picture positive. Usually, the shape of the image is assumed to be linear, that is, the verticality, distance ratio, and parallelism of all line segments can be guaranteed unchanged. Based on the assumption of linear deformation, the relationship between the coordinate on the deformed image and the unchanged m, n can be described by equations (8) and (9).
It can be seen that if the three-point correspondence between the changed image and the unchanged image is known, the equation system can be obtained at the same time, and the 6 unknowns in the equation can be obtained. That is: keep the correspondence between the distorted image and the unshaped image. Since this paper solves the transformation square matrix from the index coordinate system to the camera coordinate system during the recording process, it is only necessary to transform the square matrix.
3.5. Template Matching
The easiest way to do pattern recognition is to predefine typical patterns of known classes. When inserting a pattern, it is determined that the input pattern is closer to one of the standard patterns, where the pattern refers to the image [24]. The description of proximity requires a scale, which is usually described by the vector rule. Assuming that the input mode is a, the template of the ω class, that is, its standard mode is , and the Euclid distance is used to measure its closeness.
The input pattern x is identified as belonging to the class where 1 is the smallest. When the shape of the input pattern is the same, and the shape of the input pattern is the same but the grayscale is different, x and will be different. With this in mind, the cosine of the angle between x and is used as a measure for the actual matching process.
That is,
The closer x is to , the closer its standard similarity is to 1. A commonly used measure is the Hausdorff distance, which is defined as follows: For two finite point sets and , then M, the Hausdorff distance between M is given meaning:Where h (M, N) is called the directed Hausdorff distance between M and N,
If the distance from a point m to a point set N is defined as the minimum distance from point m to each point in the set of N point value, that is:
Then is the largest set of distances from all points in point set M to point set N.
3.6. Design of Kalman Filter
The Kalman filter is an algorithm for linear minimum variance error estimation of the state sequence of a dynamic system. The discrete Kalman filter evaluates the process using a form of feedback control: the process state is evaluated on a temporal filter, and feedback is obtained through (noisy) measurements [25, 26]. The time update equations can be thought of as prediction equations, while the measurement update equations can be thought of as correction equations.
First, define as the preestimated state quantity (pregiven quantity) of the kth step and define as the postestimated state quantity when the measurement value is given, where K is a gain or a coordination factor.
Projected future state:
The error variance is as follows:
The Kalman gain is as follows:
The updated estimate is as follows:
The error variance is as follows:
Before Kalman filter tracking, the motion model of the target should be established. The X state vector of the Kalman filter in this paper is a four-dimensional vector, namely , which shows the coordinate value or velocity value in the X and Y directions of the center of the viewing window. The linear state transition difference equation is given by:
Since only the position of the center of the tracking window can be observed on the image, the observation state vector is defined as: . Since the target is assumed to be moving at a uniform speed in unit time, the measurement equation is given by:
4. Experiment Design of Live-Action Stop-Motion Animation
The augmented reality stop-motion animation of wearable digital media needs to capture the real scene through the camera, and complete and present the superposition of the virtual information and the real scene. Then make a virtual character, and finally merge the virtual character into the real scene and display it. Finally, it is presented to the audience through augmented reality equipment.
4.1. Experimental Data
To test the promotion effect of augmented reality technology on live-action stop-motion animation, this paper selects 4 sets of stop-motion animation videos from a certain movie, of which 1 and 2 are traditional live-action stop-motion animations, and 3 and 4 are augmented reality live-action stop-motion animations. To ensure the accuracy of the experiment, the characters of the selected movie clips are all animals. The number of video frames is between 400 and 1000, the video duration is between 30 and 60, and the video pixels are . The specific data are shown in Table 1. 6 audiences were randomly selected for the test to observe whether the audience’s sense of experience was improved. The specific data are shown in Table 2.
4.2. Experimental Process
The fluency, accuracy and error rate, playback volume, and user satisfaction of traditional real-life stop-motion animation and augmented reality stop-motion animation were compared to observe the improvement of real-life stop-motion animation using augmented reality technology.
5. Experiment Results of Live-Action Stop-Motion Animation
5.1. Comparison of the Fluency of Live-Action Stop-Motion Animation
To ensure the smoothness of the augmented reality stop-motion animation, 4 sets of videos were released on 6 entertainment platforms to observe the smoothness of the augmented reality system’s live-action stop-motion animation to confirm whether there would be a freeze. Comparing the real-life stop-motion animation of the augmented reality system with the traditional real-life stop-motion animation, the results are shown in Figure 7.
The results show that on 6 entertainment platforms, augmented reality stop-motion animation is more fluent than traditional live-action stop-motion animation, especially the 5th entertainment platform. The average smoothness of traditional live-action stop-motion animation is 899 ms, and the average smoothness of augmented reality stop-motion animation is 1127 ms, an increase of 228 ms.
5.2. Accuracy Comparison of Live-Action Stop-Motion Animation
To ensure the accuracy of the augmented reality stop-motion animation, simulation experiments were carried out on 4 sets of videos to observe the difference between the accuracy of the augmented reality system’s real-life stop-motion animation and the accuracy of the traditional real-life stop-motion animation. The results are shown in Figure 8.
The experimental results show that the accuracy rate of augmented reality stop-motion animation is 3 percentage points higher than that of traditional live-action stop-motion animation, but the error rate of augmented reality stop-motion animation and traditional live-action stop-motion animation is not much different.
5.3. Comparison of the Playback Volume of Live-Action Stop-Motion Animations
Publish the 4 videos in the experiment to a website, and record each video at 8: 00, 10: 00, 12: 00, 14: 00, 16: 00, 18: 00, and 20: 00. Amount of playback. The result is shown in Figure 9.
The results showed that the total playback volume of augmented reality stop-motion animations is much higher than that of traditional stop-motion animations, and the two live-action stop-motion animations have a significant increase in the period of 12: 00 due to time factors. However, the total playback volume of augmented reality stop-motion animation increased from 9,000 to 38,000, the traditional live-action stop-motion animation increased from 8,000 to 25,500, and the total playback volume of augmented reality stop-motion animation was 94,000, and the playback volume of traditional live-action stop-motion animation was 52,000. Audiences are far more fond of augmented reality stop-motion animations than traditional live-action stop-motion animations.
5.4. Comparison of User Satisfaction
Six random viewers watched videos 1 and 2 of traditional live-action stop-motion animation and videos 3 and 4 of augmented reality stop-motion animation, respectively. Then the satisfaction score is carried out, and the result is shown in Figure 10.
The total satisfaction score is 10 points. It can be seen from the comparison that the real-life stop-motion animation effect of augmented reality is more satisfactory. The results showed that the average user satisfaction with traditional live-action stop-motion animation is 7.22, and the average user satisfaction with augmented reality stop-motion animation is 7.45. Compared with its traditional live-action stop-motion animation, user satisfaction with augmented reality’s live-action stop-motion animation increased by 3.1%.
6. Conclusions
This paper analyzed the main applications of mobile digital media and the charm of augmented reality technology, summarized the advantages of stop-motion animation based on digital reality, and made a specific analysis of mobile augmented reality animation based on mobile media. Design analysis and practical design of augmented reality animations based on wearable digital media. The reality of stop-motion animation has many similarities and differences from traditional live-action stop-motion animation. Augmented reality stop-motion animation is a kind of animation that combines virtual and real. Not only to convey information to the audience but also to allow the audience to interact with the cartoon characters. Augmented reality stop-motion animation is not limited by screen size, and viewers can watch the animation in 360°. Augmented reality animation differs from traditional animation in terms of media, delivery methods, and applications. The audience is more satisfied with the augmented reality stop-motion animation, which is more interactive and the audience is no longer passive. Augmented reality animation design methods include the use of creative design methods, point-of-interest design methods, story-based design methods, and needs-based design methods. Applying these design methods can lead to a better understanding and appreciation of augmented reality technology.
Data Availability
No data were used to support this study.
Conflicts of Interest
The author declares that there are no conflicts of interest with any financial organizations regarding the material reported in this article.
Acknowledgments
This work was supported by the Virtual Simulation Experiment Teaching Project, the Xiamen Institute of Technology (No. XJKC21018).