Abstract
The motion of a real object model is reconstructed through measurements of the position, direction, and angle of moving objects in 3D space in a process called “motion capture.” With the development of inertial sensing technology, motion capture systems that are based on inertial sensing have become a research hot spot. However, the solution of motion attitude remains a challenge that restricts the rapid development of motion capture systems. In this study, a human motion capture system based on inertial sensors is developed, and the real-time movement of a human model controlled by real people’s movement is achieved. According to the features of the system of human motion capture and reappearance, a hierarchical modeling approach based on a 3D human body model is proposed. The method collects articular movement data on the basis of rigid body dynamics through a miniature sensor network, controls the human skeleton model, and reproduces human posture according to the features of human articular movement. Finally, the feasibility of the system is validated by testing of system properties via capture of continuous dynamic movement. Experiment results show that the scheme utilizes a real-time sensor network-driven human skeleton model to achieve the accurate reproduction of human motion state. The system also has good application value.
1. Introduction
With the development of computer science, 3D display technology has become increasingly mature; human motion reconstruction technology has broad application prospects in areas such as game, film and television design [1, 2], medical science [3, 4], and sports training [5, 6]. In the field of game design, many major sports games, such as “3Dbasketball” and “NBA,” have combined human motion capture with virtual reality technology, so objects have become truly vivid and lifelike. In the film industry, many films, such as “Avatar” and “The Adventures of Tintin,” predominantly use human motion capture and virtual reality technology. In the field of medical science, the technology is also widely used in orthopedics, surgeries, physical therapy, and other areas to help doctors fully understand patients’ condition through a comparison of the motion attitude of patients before and after rehabilitation. In the field of sports training field, real-time monitoring of athletes’ training state is achieved through the display of a real-time tracking interface in athletes who wear motion capture equipment, such as wearable sensor vests [7].
At present, the common motion capture systems are based on either optical or inertial sensors. In general, optical systems are used for large game production, film shooting, and other applications that require high accuracy. However, they have high demands on the environment and are also expensive. Because of their relatively short history, motion capture systems that are based on inertial sensors have yet to be fully developed. Given their low cost and close connection with the development of information technology features, however, such systems have become a new research field in human–machine interaction.
Motion capture, as the name implies, is “the reconstruction and simulation of the test object’s motion behavior.” In scientific jargon, the model can be repeated to test the movement of an object through acquisition of the motion data of the test object and mapping of these data onto the motion capture focuses on the interaction process between humans and computers. How to interact, the use of any medium, how to facilitate object interaction, building the interactive time, and other issues are important to consider, because these issues determine the merits of the system. A number of challenges were confronted by scholars who focused on interactive algorithms and real-time system performance. Exceptions are world-renowned companies, such as “3dSuit” and “Xsens”, which have developed excellent-performing motion capture systems. Based on trade secrets and other issues, however, the core of the algorithm remains a major issue that affects not only the accuracy of the motion capture system but also its real-time properties. Basing on existing algorithms, this study analyzes in depth the data captured by sensors and establishes a data-driven algorithm model from sensors to human models. We successfully facilitate the interaction between human and model. The real-time properties of the system are also considerably improved compared with those of the traditional system.
The selection model is also closely linked with the merits of the system. The performance of different models may also be different. At present, the human body model is mainly categorized into the rod model, solid model, surface model, and multi-level model [8]. The rod model selects a limited rigid segment linking with a joint that lacks fidelity; the solid model simulates the structure of the human body through simple, solid graphics that requires a large amount of calculation and has poor stability; the multi-level model includes the skeleton, muscle, and skin layers that have high complexity and require more computation; and the surface model is composed of skeleton and skin layers that are easy to realize and require a small amount of calculation [9]. In this study, we propose a method based on surface model theory to construct a 3D human skeleton model.
The process and results of the experiment are described in this study. In Section 2, starting from rigid body dynamics, we analyze the structure and design the hardware system. In Section 3, the human motion data based on the polymerization mechanism is analyzed, and the attitude of the reconstruction process is described according to the algorithm. In Section 4, the experimental results are presented, and theoretical verification is conducted. In Section 5, the experiment is discussed further.
2. Related Work
Motion capture has a relatively long history, as reflected in the old Chinese saying “Set up a pole and see its shadow”; the meaning of this sentence is to judge the movement of the pole based on the shadow through solar light irradiation; in fact, the process is a way to capture the attitude of an object, but such a process still considerably depends on the original light imaging. With advancements in science and motion capture technology, the development from original light imaging to an optical and inertial sensing level-based system represents a major leap forward. Optical motion capture usually obtains the trajectory of human motion data through high-speed cameras that shoot continuous image sequences of the human body; the target unit is required to be equipped with the relevant features of the spot to complete the monitoring and tracking processes. A previous study [10] described optical motion capture in detail. Inertial motion capture involves human motion capture data obtained from the subject who wears limb sensor nodes on the target unit. As a result, the motion model achieves motion reconstruction. In the present study, we focus on the problem of inertial motion capture.
2.1. Attitude Algorithm
In a motion capture system based on inertial sensing, the human body structure can be simplified to a number of rigid bodies connected by a hinge joint that consists of a more rigid body structure through real-time estimates of each rigid body posture. The real-time movement of the human body is then described, so parts of the body pose estimation serve as the basis of motion capture based on inertial sensing.
Attitude estimation of the human limb is primarily achieved by placement of the sensor in the surface of the rigid body. Miniature inertial sensors are found in a three-axis accelerometer, three-axis gyroscope, and three-axis magnetometer that can be considered as gravity vector under static or low speed; three-axis accelerometer can be used as an inclinometer to determine the relative direction of the horizontal plane. The magnetometer measures the magnetic field vector under the sensor coordinate systems, and the principle followed is similar to that of a compass, which can be used to determine the rotation around the vertical axis. The gyroscope is used to measure angular velocity, and the angle can be obtained through the angular velocity integral. Acceleration and the magnetic field are complementary to the angular velocity, so the data of accelerometer and magnetometer can be used to eliminate the drift by integration of the angular velocity.
However, the acceleration and the magnetic field strength may be disturbed in different situations. When accelerometers measure motion acceleration and gravitational acceleration, motion acceleration is negligible in high-speed situations. Ferromagnetic materials or electronic products can disturb the surrounding magnetic field and form magnetic vector bias. These two issues have a major effect on attitude estimation.
Many domestic and foreign scholars have proposed different methods to address these problems. For instance, Luinge et al. [11] drifted the angle to the horizontal only by using microgyroscopes and accelerometers but did not eliminate the rotational drift on the vertical direction. Foxlin [12] described two navigation systems based on inertial and magnetic sensors that were used for commercial purpose. Roetenberg et al. [13, 14] proposed a compensation Kalman filter algorithm, in which he integrated a compensation model on the basis of the error model to offset the effect of changes in the magnetic field to the attitude estimation. However, the algorithm model calculation was complicated, and high-speed movement acceleration interference was not considered. Bachmann [15] proposed a Kalman filter algorithm based on quaternion, and he estimated the direction by using the low-frequency part of acceleration and magnetic field intensity; he also measured attitude with the angular velocity at high speed. Yun and Bachmann [16] used the QUEST algorithm to obtain a quaternion and thus estimate the attitude with acceleration and magnetic field vectors. Further integration with the angular velocity was conducted, and a linearized observation equation was used; however, a higher complexity than that of the QUEST algorithm was achieved. Later, Yun et al. used the factor quaternion algorithm (FQA) [17] to replace QUEST and, as a result, the computational complexity of the algorithm was reduced. However, the removal of the QUEST algorithm still did not address the problem on the magnetic field and the interference of acceleration.
We attempted to reduce the impact of the magnetic field and the acceleration of attitude on the basis of various postures in the reference fusion algorithm.
2.2. Motion Model Construction Method
In human motion capture techniques, the human body model is a graphical description for the human form by abstraction of the human body to simulate or reproduce the action of the human model; with this approach, the purpose of this study can be achieved with the use of human motion tracking data acquired from the device. The goal of the inertial sensing motion capture system is to utilize motion sensors for the collection of data on the human body in order to drive the established human model and thus achieve an approximate simulation of human motion. The establishment of a human motion model that is in line with the characteristics of human behavior is therefore important for the vivid simulation of human action.
According to the data collection methods and the direction of motion analysis, the construction methods of the human model in the human motion capture system are usually different. Generally, however, the human motion model can be classified as 2D or 3D based on different spatial properties.
A 2D human model refers to the calibration of human motion and a description of the movement of the limbs. This model can be further classified into the 2D sticks and 2D regional models.
The 2D sticks model uses geometry to represent the skeletal structure of the body. For example, a straight line can be used to represent bones, whereas points represent joints. Karaulova et al. [18] developed a stratified human skeleton model with straight lines and points in a human body motion tracking system on the basis of monocular video sequences. The highly simplified description of the human body in the model makes it suitable for video-based motion capture sequences. However, the lack of representation of the human form makes the model lose its verisimilitude.
The 2D regional model refers to the use of a 2D region to represent a certain part of the body in the analysis of motion capture based on image sequence. Leung and Yang [19] used the strip area to represent a part of the human body in constructing a human body model in a human body contour marking system. The 2D regional model generally applies to the extraction of the characteristics of a region in terms of human motion in graphic sequence. However, this method causes considerable information loss in the process of image processing.
3D human body model calibration refers to the human body limbs in human motion in 3D space and to the description of gestures. The model can generally be categorized into the 3D geometric model and the 3D mesh model.
The 3D geometry representation method refers to the use of some basic geometric figures to complete the construction of the human body model in 3D space. Wachter and Nagel [20] established a 3D human body model by using an elliptical cone in human motion capture based on monocular video sequences. Remondino and Roditakis [21] used lines and points to represent human body bone joints and the human skeletal system, respectively, as well as the ellipse to represent the 3D human body model in a single image or in monocular video sequence human motion reconstruction.
The 3D mesh model is defined according to the surface characteristics of the human body. It builds on the model of the human body through many facets and grids. Using this method to establish a model of the human body results in a lifelike outcome because the method can vividly describe outer human characteristics. Sminchisescu [22] proposed a construction method by using an elliptic grid for the human body model to establish a human model and facilitate motion reconstruction based on monocular video sequences. Theobalt et al. [23] designed a human-level model represented by triangular meshes to strengthen the outline of human motion capture. The goal of the inertial sensing motion capture system is to capture the attitude of each limb through inertial sensors and use the motion data to drive the human model to achieve motion tracking. Establishing a 3D model of the human body is therefore necessary.
Although the 3D mesh model can vividly describe the contour features of the muscle and skin, the muscle deformation will occur when the body is in motion, and this phenomenon increases the difficulty of graphic reconstruction. To address this situation, this study constructs a 3D human skeleton model that completely matches the human body biomechanical characteristics of the human body model. We also use kinematics theory to restrain and limit the development of the skeletal model and thus achieve accurate limb posture features during human movement.
3. Attitude Calculations
In this section, we describe the solution to the attitude of human motion through sensors. Initially, we select the types of sensors and consider the means of communication between hardware. Then, we calculate the attitude of a single node by combining sensor data. Finally, we calculate the attitude from a single node to the entire body according to the movement mechanism of the human skeleton.
3.1. Data Collection and Communication Network
The inertial sensor-based human motion capture system examined in this study contains an attitude acquisition module, an information processing module, and an attitude reconstruction module. The attitude acquisition module combines the mpu-9150 (integrated accelerometer, gyroscope, and magnetometer) microsensor with nrf-51822 (BLE 4.0 ultra-low-power module), mpu-9150, and nrf-51822, as can be seen in Figures 1 and 2. The module forms a small sensor network between each node. We also set two sink nodes to receive the transmission data from the subnode. The sensor network configuration is shown in Figure 3. To communicate with the host computer terminal, we use Bluetooth technology; the transmission distance is about 10 meters.
3.2. Single Node Attitude Solution
In the process of human motion, each joint wears a sensor node, and the attitude calculation algorithm of each node consists of the following steps.
Step 1 (sensor calibration). For the gyroscope, the main work is to remove zero offset, which is to calculate the zero bias through many times’ weighted average and make the initial value minus zero offset. For the accelerometer, at first, we set low-pass filter to eliminate jitter effects, and then using the weighted average filter to eliminate jitter effect further. The calibration of the magnetometer involves two aspects: hard offset and soft offset. Hard offset is mainly caused by the hard magnetic materials. It shows that the scatter diagram of random rotation magnetometer readings is a spherical shell away from the origin. The center of the spherical shell is the vector value of the hard offset. In this system, we obtain the hard offset through a large number of samplings and calculation of the minimum residual value. Soft offset is mainly caused by soft magnetic materials. It shows that the scatter diagram of random rotation magnetometer readings is an ellipsoidal shell centered at the origin. The value of the soft offset can be represented in a matrix; through matrix solution, the soft offset can be calculated. The calibration of the magnetometer is therefore achieved. To reduce the noise-induced jitter, a low-pass filter must be set to filter the data of the accelerometer and magnetometers.
Step 2 (the angular velocity integral). We measure angular velocity information in the process of human motion by using a gyroscope. With integral operation on the angular velocity, the angle component in the process of motion can be obtained. Using a first-order high-pass filter followed by a first-order Taylor expansion method, we finally obtain initial quaternion information of the human body in the process of movement.
Step 3 (solution attitude of the accelerometer and magnetometer). We use FQA algorithm to transform the acceleration value and the magnetic field information into the attitude value. The advantages are as follows. First, the magnetometer data have an effect on the heading angle only (Yaw). Second, using the algorithm significantly reduces the effect of the magnetic field on the attitude computation. Third, the amount of calculation is considerably reduced. The FAQ is also limited, but the use of an ideal environment means no linear acceleration exists, and if it does, it is only under the effect of the magnetic field. This condition means that the FAQ results in little disturbance in linear acceleration and magnetic field interference.
We conducted a verification experiment. We placed the inertial sensor model in static state and observed the attitude calculated by the FAQ algorithm. Then, we repeat making a ferromagnetic material away and close to the sensor module and continue to observe the changes in the moment waveform. Figure 4(a) presents the attitude angle calculated by the FAQ algorithm when little magnetic interference exists. Figure 4(b) presents the attitude angle calculated by the FAQ algorithm when large magnetic interference exists. From the chart, we can see that in the case of nonlinear acceleration, the roll angle and the pitch angle calculated by the FAQ algorithm have no distortion with or without magnetic interference. Magnetic interference only has an effect on the Yaw angle.
(a)
(b)
Step 4 (sensor data fusion). We fuse the data of the sensors. In summary, in the case of linear acceleration or a large magnetic field, only the attitude information obtained through the angular velocity integral works. With the uniqueness of the representation of the direction cosine matrix considered, the opposite case sets the weight and average value for the attitude value obtained from the two preceding methods. In slow motion, the two move cooperatively; during strenuous exercise, only the angular velocity integral works.
The above process can be summarized as a complementary filter. The single node attitude solution process is shown in Figure 5. By using the attitude calculation of a single node with the above algorithm, we can obtain the attitude data of each node. In the actual process, we commonly use the rotation matrix to represent the rotation of a 3D object. Many methods can be used to express the rotation matrix. The most common ones are the Euler angle and quaternion. However, the Euler angle involves the gimbals problem, so we use the quaternion to represent the rotation matrix in the actual process. To understand the process of the experiment, we converted the quaternion to the Euler angle. Then, we choose the palm as the wearing node and observe the effect of the curve motion.(a)We place the palm on the level direction; this is step 1. After a smooth, palm repeats the movement: vertically bends down to a certain angle and returns to the horizontal state, this process is considered as step 2. After step 2, the palm is bent vertically up to a certain angle and then is returned to the horizontal position; this is step 3. The movement process is shown in Figure 6(a), and the attitude motion curve is shown in Figure 6(b).(b)We place the palm to be stable on the level direction; this is step 1. The palm is rotated right to a certain angle along the arm axis and then is returned to the horizontal position; this process is considered as step 2. After step 2, the palm is rotated left to a certain angle along the arm axis and then is returned to the horizontal position; this is step 3. The movement process is shown in Figure 7(a), and the attitude motion curve is shown in Figure 7(b).(c)We place the palm on the level direction; this is step 1. The palm is bent right to a certain angle along the arm axis and then is returned to the horizontal position; this process is considered as step 2. After step 2, the palm is bent left to a certain angle along the arm axis and then is returned to the horizontal position; this is step 3. The movement process is shown in Figure 8(a), and the attitude motion curve is shown in Figure 8(b).
(a)
(b)
(a)
(b)
(a)
(b)
With the motion curve, the designed algorithm of the experiment solves the interference problem of the magnetic field and the acceleration in the attitude computation. The attitude calculation accuracy can meet the requirements.
3.3. The Body Attitude Solution
The cooperative motion of various body limbs is involved in the process of human motion tracking. Analysis of the human body kinematics principle indicates that the motion between various nodes involves related links. Therefore, calculating the attitude of the entire body is necessary.
3.3.1. Posture Initialization Calibration
In the process of using sensor nodes to capture human motion, three coordinate transformations are involved: inertial sensor coordinates, geomagnetic coordinates, and human skeleton coordinates. The inertial sensor coordinate is set as the calibrated coordinate system for the chip itself. The establishment of geomagnetic coordinates is due to the fact that the data on each axle of the magnetometer determine the magnetic field distribution detected by each axis. However, according to the distribution of the magnetic field of earth, the world coordinate system can be defined as the geomagnetic coordinates (the origin is the center of the earth, the -axis is along the equatorial plane pointing outward, the -axis is along the equatorial plane with a right orientation, and the -axis points to the north). The human skeleton coordinates will be defined in the following article. During the experiment, we must initialize the pose calibration every time in the testing process; that is, calibrate the motion of the joint body of the experimental individual and obtain the rotation matrix of any part of the human skeletal coordinate system from the sensor coordinates of this bone because of the difference between the wearing position and the individual. We take the limb system as an example. According to the sensor node programming, the lower limb needs three sensor nodes, which are foot , leg , and thigh , respectively, to conduct the initialization calibration. Figure 9 shows the lower limb structure diagram. The calibration process is as follows.
(1) Lower limb toward the lateral stretch: with the static action at this time, the acceleration of gravities and can be obtained from a lower leg sensor node and foot node, and then the rotation matrix of of the leg and of of the foot can be calibrated:
On the basis of step 1, turn the lower limbs forward about 90° from rear of your body, then turn about 90° backward from front of your body. From the dynamic posture, in the lower limb movement process, we can obtain the output angular velocity readings and and the reverse readings and , which correspond to the thigh, leg, and foot sensor nodes. Therefore, the rotation matrix of of the foot and of of the leg can be calibrated as follows: The rotation matrix for the foot and the leg can be further obtained as follows:
(2) The lower limbs shall be upright initially, then extend them right ahead about 90°, with feet being perpendicular to the horizontal plane. We then obtain the thigh angular velocity , so the thigh rotation matrix of can be calibrated as follows:
(3) The lower limbs shall be upright initially, then lift the lower limbs backward at a constant speed until they are nearly perpendicular to the trunk, that is, return to the state in Step (1). According to this position, we can obtain the acceleration of gravity in the thigh when it is in the upright position. When the leg is raised to the front, we can obtain the acceleration of gravity in the thigh at the moment, which in turn obtains the thigh rotation matrix of at the moment: At the same time, we obtain the rotation matrix :
3.3.2. Real-Time Calibration Attitude in the Human Motion Process
To obtain an accurate position of the human skeleton model with the use of inertial sensors to measure the data of human motion, we need to calibrate the relative position between the attitude measurement unit and the human body movement between moving limbs. The calibration process aims to determine the rotation matrix between the inertial sensor coordinate system and the measured body coordinate system and between two movements of the limbs. From the perspective of rigid body dynamics, the two connected segment bones are considered a rigid structure. For example, the No- and No- bones are connected, and then the attitude solution is determined. The rigid structure of the connected skeleton is shown in Figure 10.
Suppose is the rotation matrix of the sensor coordinate system that corresponds to the bones compared with the geomagnetic coordinates. is the rotation matrix of the bones with respect to their layout of the sensor coordinate system. can be calculated through the initial data measured by the sensor, and can be obtained in the initialization calibration process, which can calculate the attitude matrix of the bone in its coordinates: The rotation matrix from bone to bone is From formula (7) and (8), we can obtain
Suppose is the transformation matrix from No- bone to No- bone of the rigid body. From the coordinate nature of the conversion matrix, we can obtain the following: where is the translation matrix from No- bone to No- bone, and each element of is the true 3D data of the No- bone. In this study, the translation vector from the sensor coordinate system to the body coordinate system, as well as the coordinate translational component from the geomagnetic to the sensor node coordinate, is the upward component obtained from the acceleration value integration.
The transformation matrix multiplication process proposed by Craig [24] indicates that the transformation matrix in the No- bone corresponding to the first bone that can be calculated:
According to the position of the first sensor node, the transformation matrix of the No- bone can be calculated. In the experiment, however, we can measure and calculate the spatial coordinates , , and of the pelvic area in the model: where denotes the vector in body and is the corresponding limb length; the recurrence relations are as follows:
The vector of each limb can be obtained by order recursion. Therefore, the attitude and coordinate of the No- bone can be computed according to the root node coordinates; virtual human motion reappearance can finally be implemented.
We can obtain the vector of each limb, followed by recursion. Therefore, the attitude and coordinate of the No- bone can be calculated according to the coordinate of the root node. Finally, reproducing the motion attitude of the virtual human is realized.
4. Model Construction
The human body is a complex life system, and its exercise system consists of more than 200 bones, more than 600 muscles, and countless nerves commanding these bones and muscles; in this regard, specifying each piece of bone in the experiment is impossible [25, 26]. Basing on the idea of the surface model and in view of the human body posture data collected with the microsensor, we propose a method to construct a 3D human skeleton model.
4.1. Hierarchical Human Body Model Based on Constraints
In the literature [27], the automatic estimation of the hierarchical topological structure human skeleton model was described; a detailed induction to the motion of the skeleton on the basis of the biomechanics principle was performed. In the present study, we took the topological structure of the model and divided the human body into 16 bones and 14 joints. Bone segments are connected by joints. We defined the relationship between the joints as the relationship between parent and son. Parent node movement will lead to relevant subnode movement. In the definition of the hierarchical structure in the human body, the pelvic node is the center of gravity of the human body and is also the reference node of the entire body. The pelvic node extends upward and downward and traverses the entire skeleton. The skeleton hierarchical structure is shown in Figure 11(a). Each joint of the human body presents angle constraints, so Baerlocher [28] presented a clear definition: the joints in different bones of the body are categorized as rotary, hinge, and universal joints. Rotary joints have only one direction of freedom, and their rotation range is limited to a certain extent. Hinge joints have one degree of freedom, and they can twist and varicose around other joints. Universal joints have two degrees of freedom, and they have a structure similar to that of a ball groove. They can move around the ball groove structure. Table 1 shows the types and scope of activities of human body joints.
(a)
(b)
4.2. The Coordinate System of the Human Skeleton
This study presents a coordinate system of the human skeleton based on joint tree structure through analysis of the human skeletal hierarchy and the principle of 3D animation structure. The pelvic joints are taken as the reference points, and the coordinate system of other skeleton is calibrated according to the forward kinematics principle. The skeletal coordinate system after calibration is shown in Figure 11(b).
4.3. The 3D Models of Motion Capture
Based on the above relationship, we designed a skeletal motion model consistent with human motion by loading a human skeleton file based on Microsoft Visual Studio2012 platform and OpenGL development library environment. Furthermore, we can adjust the skeleton size and save the motion data of each bone. Finally, we designed the terrain, so that the effect is realistic. Figure 12 describes the 3D human skeletal motion tracking system.
5. Results and Discussion
To validate the 3D human model that simulates real human motion, we compared the key frame curve extracted from the real human motion video image [29] with a gesture curve when the skeleton simulates relative movement.
In the experiment, we took the up limb system as an example and shot the video sequence of continuous movement in the process of bending of human arms. Through image processing, we extracted the key frames curve of the elbow bending angle during the arm bending process and compared it with the elbow bend angle data curve of the real human body simulating the corresponding action. According to their fitting degree, we can judge the accuracy of the model.
We designed an arm motion tracking verification platform based on the algorithm described in this study. In the course of the experiment, we dressed the sensor node in the elbow and tracked the real-time motion of the elbow. The image sequence is based on the plane, so we tracked the bending angle of the arm in the direction of the vertical plane and recorded the motion process with a high-speed video camera. By contrast, in the visual motion video images and 3D motion effects, we can evaluate the motion tracking effect. By extracting the blending angle through the arm flexion extension in the video sequence and by comparing the output angle of the sensors, we can evaluate the effect of the system according to the data point. Figure 13 shows the comparison of the effect of motion tracking and the video image, and Figure 14 shows the comparison of the extracted output from the sequence of video frames and the actual output angle of the sensors.
Figure 14 depicts the fact that two curve fittings can basically meet the requirements; the angle error is less than 1 degree. Therefore, the model has some reference value.
To evaluate the performance of the entire system, actual human testing was performed. We tested the motion tracking effect vis-a-vis the real motion of the human body by placement of sensor nodes on all specified parts of the human body. We took a photo on the effect of motion tracking in the experiment with a high-speed camera and conducted some real-time motion capture.(1)With the arm motion process taken as an example, when the system has stabilized after wearing during the experiment, the right arm does the movement, as shown in Figure 15(a). The system can accurately track the motion trajectories of the arm. Figure 15(b) shows the system dynamic trajectory tracking of the arm.(2)In the preceding process, the motion capture of a single arm is described. However, we need multiple motion joint investigations to assess the reliability of the entire system. In this step, we conducted the motion capture test on the cooperative motion of two arms. With the waving motion in life taken as an example, Figure 16 describes the capture process in a real human hand waving action.(3)Finally, with the entire human body movement taken as an example, we captured the motion from still standing to turning and kicking. Figure 17(a) shows the real human motion, and Figure 17(b) shows a set of the motion capture effects.
(a)
(b)
(a)
(b)
With human soft tissue jitter and other relevant errors considered, the model can accurately track the real-time motion of the human body through the dynamic tracking of actual human bodies.
6. Conclusions
In this paper, starting from the movement mechanism of the human body skeleton, we constructed a new attitude calculation method based on inertial sensing theory. We also verified the attitude calculation method through the movement of hand joints. Then, we established a 3D human motion model according to hierarchical modeling theory based on computer graphics. Finally, we investigated the performance of the system by taking photos of the effect of motion capture with a high-speed camera. The experimental results show that the scheme can effectively realize the real-time motion tracking of the 3D model of the human body.
However, the system still has some limitations even if it can capture human motion effectively. In the process of communication, the motion data are packed together and transmitted to the host computer, but time loss inevitably occurs in the processes of transmission and attitude calculation. Our calculation demonstrates that the transmission of the entire data package to the host computer takes about 19 ms, but the refresh time for the system interface has a time delay of about 1 ms. Therefore, the refresh time of the entire system needs about 20 ms. In the real world, however, if the refresh rate is higher than 40 ms, the human eye can easily detect the delay. The 20 ms delay is thus acceptable for the human eye despite some hysteresis. Of course, the accuracy of the sensor also affects the performance of the system. For example, the sensors of motion capture systems, such as “3Dsuit” and “Xsens MVN,” are of high precision, high sensitivity, and fast reaction speed, but these systems are relatively expensive. Taking cost into consideration, our experiment selected the mpu-9150 sensor whose sensibility has a certain gap with that of the special inertial sensor. When the sensor is in high-speed motion, the response will be somewhat hysteretic, but this does not happen if the sensor is in low-speed motion. Therefore, the key to addressing such a problem is to choose sensitive inertial sensors and to further improve the algorithm. However, the model selection also needs further improvement. The capture effect of the system can meet the development requirements of some small interactive games.
Conflict of Interests
The authors declare no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors gratefully acknowledge the financial support of the National Natural Science Foundation and the Natural Science Foundation of Jiangxi Province. The authors would also like to thank the editor and anonymous reviewers for their useful comments that helped improve the quality of this paper.
Supplementary Materials
We provided some related videos which described the process of the sensor network driving the human skeleton and the motion capture effect of some daily movements. The processes of sensor driving limbs were demonstrated in the videos. Of course, the performance of the system is to be perfected.