Abstract
On the basis of inertial navigation, we conducted a comprehensive analysis of the human body kinematics principle. From the direction of two characteristic parameters, namely, displacement and movement angle, we calculated the attitude of a node during the human motion capture process by combining complementary and Kalman filters. Then, we evaluated the performance of the proposed attitude strategy by selecting different platforms as the validation object. Results show that the proposed strategy for the real-time tracking of the human motion process has higher accuracy than the traditional strategy.
1. Introduction
Motion capture is a multidisciplinary application field that features the interaction of recently developed technology for different areas, such as electronics, communication, control, computer graphics, human ergonomics, and navigation. Moreover, motion capture is currently a research hotspot in the field of human–computer interaction and has thus undergone extensive development in film and television production [1], robot control [2], interactive games [3], sports training [4], medical rehabilitation [5], and other fields.
Traditional motion capture techniques are mostly extended on the basis of optical properties and are used to drive a virtual character model to achieve refactoring movement after being filmed and processed in a professional studio by multiple high-resolution cameras. The common optical systems are Vicon [6] and BTS [7]. Such systems can generally provide high accuracy, but their price is relatively high for public use. Hence, these systems are commonly used in scientific research institutes, large laboratories, and other organizations. With the development of miniature sensing and navigation technologies, motion capture technology based on motion sensors has become possible. At present, the use of motion capture systems, which consist of multiple sensor nodes attached to different parts of the body, has increased gradually, thereby combining data acquisition, processing, and communication [8–11]. Compared with optical systems, motion capture systems feature simple equipment, low cost, and strong fault tolerance.
The human body is a multi-rigid-body structure composed of multiple bones. The movement process of the human body is mainly determined through the position of the joints that connect bones. Such process mainly reflects the mutual movement between bone changes and the relative displacement of space. Human motion capture based on motion sensing obtains accurate data of two characteristic parameters, namely, joint movement angle and displacement as the center, with the sensor node collecting real-time original motion data. Angle and displacement parameters are calculated by using a related algorithm to achieve data mapping with the motion model, as well as the real-time tracking of human body movement. The principle of the model is similar to the navigation in an inertial navigation system, a key part of which is the attitude solution.
(1) Problem Formulation. The problem of improving accuracy of the real-time tracking of the human motion process has been discussed in a host of papers [12–16]. The methods proposed in these articles have addressed the problem to some extent but cause others problems as well, such as increasing the computational complexity. Obviously, accuracy and real-time performance are taking more and more significant roles in motion capture systems. So, captured motion data should be simplified to meet the corresponding requirements. In the film and video game production, the displacement information and rotation information are widely used to represent human actions. Therefore, in these paper, the parameters which can restore the original action well have been chosen, namely, displacement and angle. Then, the new filter technique combining complementary and Kalman filters has been introduced in motion data processing. Verified by experiment, the new strategy has been proved to be more robust and effective.
(2) Present Study. Several scholars have conducted related research on the attitude algorithm strategy for human body motion capture. Yun et al. [12] proposed a Kalman filter algorithm based on a quaternion, which uses acceleration and the low frequency part of the magnetic field strength to estimate the direction under low speed cases and the angular velocity to measure attitude under high speed cases. Roetenberg [13, 14] proposed a type of compensation Kalman filter algorithm by combining compensation and error models to compensate for the estimated effect of magnetic field change. However, the algorithm model is complex and does not consider the acceleration interference condition under high-speed motion. Chou [15] and other researchers used the QUEST algorithm, which estimates attitude on the basis of the quaternion obtained through acceleration and magnetic vectors and further integrates angular velocity. This type of treatment solves the observation of the linearized equation. However, the computational complexity of the QUEST algorithm is high. The author later adopted the factor quaternion algorithm (FQA) to replace the QUEST algorithm [16] and reduce the computational complexity. However, the aforementioned methods have not completely addressed the removal of magnetic field and acceleration interference.
Considering the complexity of existing algorithms and interference conditions, this study proposes a complementary Kalman filtering algorithm model. By fusing the original data from the sensor, extracting the relatively accurate characteristic parameters, namely, displacement and angle, and selecting different platforms, the accuracy of the attitude algorithm and its real-time performance can be evaluated effectively.
2. Methods
Human body movement in a three-dimensional space is mainly characterized by the angle of the relative movement between the body and the space of the change of displacement. Motion capture is the process of capturing the angle and displacement of human body movement during actual movement to determine the reoccurrence of movement. In this study, the process of calculating movement characteristic parameters is shown in Figure 1.
According to the principle of human anatomy, the human body can probably be divided into the following parts: pelvis and thigh, leg, foot, spine, head, arms, elbows, and hands. The structure of the hand is different from that of the other parts, and we thus distinguish it from the other parts. Each body part is connected by joints. We take the pelvis as the reference node, set the corresponding sensors to every part, collect human movement information, use the related algorithms, and calculate the human body characteristic parameters to track human body movement. The layout of the sensors on the body and hand is shown in Figures 2(a) and 2(b), respectively. Transformation involves three types of coordinate systems, namely, the global coordinate system (GCS), the body coordinate system (BCS), and the sensor coordinate system (SCS). In this study, we define the GCS as the -axis referring to the front, the -axis referring to the right, the -axis referring to the top, and the entire right hand coordinate system. In the BCS, each body part corresponds to the coordinate system. The SCS is defined as the coordinate system of the sensor. We assume that the BCS is the SCS after calibration to facilitate analysis.
(a)
(b)
2.1. Sensing Signal Modeling
This article selects the sensor model MPU9250 (integrated three-axis accelerometer, three-axis gyroscope, and three-axis magnetometer). According to [17], we can model the sensor signal as follows.
For the three-axis gyroscope, is the angular velocity, is the gyro drift, and is the white noise interference. Then, the signal measured by the gyroscope can be expressed as follows:
For the three-axis accelerometer, is the acceleration, is the gravitational acceleration, is the accelerometer offset, and is the white noise interference. Then, the signal measured by the accelerometer can be expressed as follows:
For the three-axis magnetometer, is the magnetic field, is the geomagnetic disturbance, and is the white noise interference. Then, the signal measured by the geomagnetometer can be expressed as follows:
The gyroscope and accelerometer offset changes can be represented by the Gaussian white noise and the covariance matrix of the first-order Markov model as follows:
Figures 3, 4, and 5 show the initial accelerometer, gyroscope, and magnetometer dynamic measurements, respectively.
2.2. Displacement Solution Strategy
We use the quaternion to represent the orientation relationship of movement to avoid the occurrence of a singular value in the process of movement. On the basis of [18] and the signal model of the gyroscope, the position relation of each limb in the GCS at time can be expressed as follows:where , is the sampling interval time, , and is the quaternion used to locate the direction at time . is a 4 × 4 skew symmetric matrix expressed as follows:where is the crossover operator. Prediction speed and displacement can be achieved through the integration of acceleration, that is,wherewhere is the gravitational vector under the BCS and is the direction cosine matrix described by the forecast quaternion . Figure 6 shows the displacement motion curve calculated at this time. where and is the crossover operator of . Although the relative displacement parameters can be computed, calculating a large amount of data with high accuracy is difficult because of the acceleration sensor drift and noise interference, as shown in the curve in Figure 6. Moreover, the integral process continues to result in errors. As such, the displacements at this point are inaccurate and feature a large data drift. In this case, such displacements require filtering. We use the complementary Kalman filter fusion process on the displacement parameters. The solution strategy is shown in Figure 7.
We establish the state error and measurement models as follows:where is the process noise of the covariance matrix and is the state transition matrix. Taking a section of the human body as an example, the state error vector can be represented by the previously presented formula, is the measurement error, is the measurement noise of the covariance matrix , and is the measurement matrix.
The real motion quaternion can be represented by the predicted value and error value as follows:
After processing according to the literature [19, 20], we can obtain a real-time update of the direction positioning error, velocity error, and displacement error value.where , , is a 3 × 3 unit matrix, and is the crossover operator of the accelerometer for acceleration after the removal of the offset.
According to [17], the acceleration and magnetic vectors can be normalized, and the input of the measured values of the complementary Kalman filter can be calculated. According to [18, 21], direction, speed, and displacement at time can be corrected. The correction equations can be expressed as follows:where , , and are the real value, predicted value, and error value of the direction positioning error, respectively, at time . , , and are the actual displacement, prediction displacement, and error of displacement at time , respectively. Finally, by updating the Kalman filter, the final updated equations of displacement can be obtained as follows:where is the displacement under the global coordinates after using the Kalman filter, is the Kalman gain, is the forecasting displacement vector, is the process noise, is the measurement matrix, and is the measurement noise, from which the ultimate displacement can be obtained.
2.3. Angle Solving Strategy
The traditional attitude algorithm strategy was developed on the basis of inertial navigation and initially involved the use of a gyroscope to obtain the integrated solutions of attitude angles. However, given the influence of integral drift and the large deviation of calculated attitude angles, the traditional attitude algorithm strategy cannot be easily used in the practical process. Hence, we combine gyroscope and accelerometer measurements to calculate gesture. However, the calculation of the navigation angle produces a relatively large drift because of the influence of the geographical field. Figures 8 and 9 show the calculation results obtained with a three-axis gyroscope and a gyroscope combined with an accelerometer.
We put forward the use of an accelerometer, a magnetometer, and a gyroscope to calculate attitude angle data. Fusing the data of gyro feedback when calculating the angle of node movement to improve the stability of posture is necessary because the dynamic response of the posture calculated with the accelerometer and magnetometer is relatively poor and is sensitive to noise vibration.
The fusion process begins with the FQA [20] to conduct a combined calculation of the data pretreated with the accelerometer and magnetometer and to obtain the initial attitude angle. We convert the Euler angle to quaternions , , , and and use them as a set of reference quantities to avoid a singular value. Then, we introduce the complementary filter to conduct the fusion calculation on posture and to obtain a stable attitude quaternion. The calculated attitude information is accurate. However, a small amplitude oscillation is still observed in the stationary process. Then, by using the transformation formula of the quaternion and Euler angle, the Euler angle of movement can be obtained. The mutual transformation between a quaternion and a Euler angle can be shown in (15). The motion curve quaternion transferring into the Euler angle is shown in Figure 10.
(a)
(b)
(c)
We use the Kalman filter to filter the attitude quaternion and consequently improve the disturbance resistance and anti-interference performance of the proposed strategy. The decoding strategy is shown in Figure 11.
In the decoding process, we define the attitude quaternion from the GCS to the human body as and set the quaternion error as follows:where . The quaternion complementary filter algorithm can be represented by the following formulas:where , , is the triaxial gyro output, and is the drift calculated with the gyroscope. When , , the quaternion attitude angle at the moment can be calculated by using the complementary filter and can be further corrected by using the Kalman filter through the model and the measurement noise matrices and . The matrix forms of and are expressed as follows:
In the experimental process, , , and . The optimal attitude angle recurrence formula at time can be expressed as follows:where is the state transition matrix, is the Kalman gain, the measurement matrix, and is the initial value. By using the Kalman filter, the optimal estimates of the angle can be obtained. Figure 12 shows a set of quaternions of the movement angle calculated with this algorithm. Through the quaternion and Euler angle transformation, the Euler angle can be obtained by using the Kalman filter. The attitude angle, compared with the complementary filter, has smaller shocks, as shown in Figure 13.
(a)
(b)
(c)
3. Results
We evaluate the performance of the algorithm by conducting the following experiments.
(1) First, in evaluating displacement, we take the foot node as the reference, select the traditional optical capture system to capture the displacement of the foot node as a reference value, and compare the measured displacement calculated with the designed algorithm and the acceleration of the quadratic integral with the displacement captured by the optical capture system. The optical capture system is shown in Figure 14. The detailed contrast displacement curve of the human body in walking motion is shown in Figure 15. The error curve is shown in Figure 16. Table 1 describes in detail the algorithm compared with the error results after the second integral process.
(a)
(b)
(c)
(a)
(b)
(c)
(2) When evaluating the characteristic angle, the algorithm we finally adopt uses the quaternion to represent such angle. However, the quaternion visual is weak and cannot be verified. As such, the transformation relation of the quaternion and Euler angle transforms the characteristic angle into the Euler angle. We then take the multiple spindle motor debugging platform equipped in the laboratory as the object. The multiple spindle motor debugging platform is shown in Figure 17. The angle of the axial rotation of the sensor is presented by the motor rotation angle and serves as a reference. We compare the characteristic and reference quantities measured by the algorithm with the characteristic quantity measured by the commonly used complementary filter. Figures 18(a), 18(b), and 18(c) show the comparison results of the calculated angles. Figures 19(a), 19(b), and 19(c) show the error curves. Table 2 describes in detail the differences between the calculation angle obtained by using the proposed algorithm and that obtained with the complementary filter.
(a)
(b)
(c)
(a)
(b)
(c)
As indicated in the validation results, the error of displacement is maintained within 0.05 m in this study. The calculated motion angle error is within 1°. The accuracy of the proposed algorithm under dynamic conditions is higher than that of the traditional complementary filter algorithm, which has several reference values.
(3) In addition to evaluating the accuracy of the algorithm, real-time performance is also an important indicator that should be considered. On the basis of the node call time of the project, a timing clock is designed to calculate the attitude updating cycle. During the experimental process, six experiments are conducted to extract data for an average of 500 times for the attitude updating cycle test. Then, the experimental results are compared with the results of the attitude algorithm cycle of the traditional complementary filter algorithm. The experimental comparison results are shown in Table 3.
Considering the influence of the Kalman filter, we can deduce that the time response of the algorithm is slightly lagging compared with that of the complementary filter. The refresh rate of the proposed algorithm is calculated to be approximately 11 ms. Although such value is higher than that of the complementary filter, the refresh rate can basically satisfy the real-time system requirement, and the effect on the entire system is not significant.
Through a comprehensive analysis of the calculating errors and real-time algorithm and by considering the weight of the calculated precision and update cycle, we can conclude that the designed human movement algorithm can basically meet important requirements.
Finally, a new type of related motion capture device is designed in this study to evaluate the practicability of the algorithm. The device is mainly composed of a sensor and gathering nodes attached to different parts of the body. The sensor node is a combination of the MPU9250 and STM32F103 modules, which are connected through the SPI interface. The sensor nodes are shown in Figure 20. The gathering node is a combination of the STM32F103 and EMW3162WIFI modules, as shown in Figure 21. The gathering and sensor nodes communicate through the CAN bus. Feature data are collected by the gathering node and sent to the PC-3D motion tracking interface through the WIFI module. By considering the data-driven characteristic parameters, the movement model can track real human body movement. The software interface of the motion capture system is shown in Figure 22. The system structure is shown in Figure 23.
(a)
(b)
Figures 24(a) and 24(b) show the track effect of the motion capture system. By comparing the driving effect of the model with the calculated characteristic parameters and real human body movement process, we find that the proposed algorithm can be used to track real human body movement and features a certain reference significance.
(a)
(b)
From the above experiments, the proposed method has been proven to have high accuracy in both displacement and characteristic angle and have good real-time performance. Of course, it is slightly inferior to the traditional optical systems in [6, 7], but its price is relatively lower and can meet the needs of the public. In [22], the author proposed an adaptive complementary filter for identifying human upper arm movements by replacing the coefficients which can be dynamically bound with a linear relationship to variables to minimize error of angle estimation. It demonstrated root mean squared error of for upper body limb orientation estimation when compared to gold standard VICON optical motion capture system. However, it will cost pretty much time to deal with data from accelerometer and gyroscope, as shown in Table 4. In our method, angle and displacement parameters are calculated by using a related algorithm, so it will save a lot of time without sacrificing accuracy and the algorithm can be simplified by combining complementary filter with Kalman filter, as shown in Table 5 and Figure 25.
4. Discussion and Conclusion
A calculation method for determining the characteristic parameters in the human body movement process was designed in this study on the basis of the distributed motion sensor motion capture platform. Compared with the traditional decoding strategy, this method is implemented from the perspective of attitude data error and real-time algorithm. Then, different platforms were selected as the experimental object, and the effectiveness of the proposed algorithm was tested. Finally, a motion capture test on the characteristic parameters extracted with the proposed algorithm is conducted to verify the performance of the proposed algorithm in terms of the effect of motion tracking. The proposed algorithm still has several deficiencies. For instance, in the case of magnetic fields, the data can suffer from a large deviation, and the motion capture model needs further optimization, for example, to generate bone and skin models. Generally, the proposed design has a certain innovation and application value and can thus serve as a new reference for the future research and design of motion capture systems.
Competing Interests
The authors declare that they have no competing interests.
Acknowledgments
This research is supported by National Natural Science Foundation of China (61663011) and Postdoctoral fund of Jiangxi Province (2015KY19).