Abstract
In the field of sports, the formulation of existing training plans mainly relies on the manual observation and personal experience of coaches. This method is inevitably subjective. The application of artificial intelligence technology to the training of athletes to recognize athletes’ posture can help coaches assist in decision-making and greatly enhance athletes’ competitive ability. The human body movements embodied in sports are more complicated, and the accurate recognition of sports postures plays an active and important role in sports competitions and training. In this paper, inertial sensor technology is applied to attitude recognition in motion. First, in order to improve the accuracy of attitude calculation and reduce the noise interference in the preparation process, this article uses differential evolution algorithm to apply attitude calculation to realize multisensor data fusion. Secondly, a two-level neural network intelligent motion gesture recognition algorithm is proposed. The two-level neural network intelligent recognition algorithm effectively recognizes similar actions by splitting the traditional single-level neural network into two-level neural networks. Experiments show that the experimental method designed in this article for the posture in motion can obtain the motion information of the examinee in real time, realize the accurate extraction of individual motion data, and complete the recognition of the motion posture. The average accuracy rate can reach 98.85%. There is a certain practical value in gesture recognition.
1. Introduction
With the rapid development of science and technology, the use of scientific and technological means to improve the quality of sports training has gradually attracted people’s attention. In the past sports training process, the trainers used the less difficult video replay and analytical management methods to explain the movement essentials to the athletes. It was not intuitive and scientific, lacked authenticity and interactivity, and could not meet the basic evaluation requirements for athletes’ movement specifications and training results [1, 2]. However, sports videos can extract the training parameters of the athletes. We can further analyze the training parameters by constructing models and observing the training actions. Get the results of the movement analysis of the athlete training [3]. Ultimately, we can ensure that athletes understand their shortcomings intuitively.
There are two main recognition methods for human body gesture recognition, namely, the recognition technology based on image analysis and the recognition technology based on inertial sensors [4, 5]. The recognition technology based on image analysis mainly collects video, image, and other information to recognize the human body posture [6]. Therefore, it is necessary to place monitoring equipment such as cameras in advance in the detection environment to collect data. Image analysis technology has been applied to human body gesture recognition at an early stage, and the technology is relatively mature. The early ones include research based on monocular video and research based on multiview video [7, 8]. In addition, Sokolova and Konushin [9] used multiple cameras to perform multiangle detection of human action poses and used neural network algorithms to train and classify image and video data. However, due to the large amount of data contained, it is difficult to realize real-time monitoring. There are still many shortcomings in the recognition technology based on image analysis, which requires high precision for the equipment, and the equipment is relatively bulky and not portable. Video capture is prone to blind spots; some places are not easy to be observed, and the monitoring range is obviously limited. In addition, the large amount of image acquisition data can easily lead to insufficient storage and fail to achieve the purpose of real-time monitoring [10]. The recognition technology based on inertial sensors makes up for the shortcomings of image recognition technology [11]. The development of science and technology has driven the improvement of sensor technology. Sensor equipment has become the best effect due to its small size, high precision, flexibility and easy wear, low environmental requirements, high sensitivity, low energy consumption, and good real-time performance. It is widely used in various fields, such as competitive sports [12], rehabilitation therapy [13], somatosensory games [14], and other aspects.
At present, there are still many shortcomings in the recognition of human motion posture, and there is a lack of relevant research in the recognition of motion posture. In response to this situation, this paper proposes an inertial sensor-based motion posture recognition algorithm. First, due to the drift in the output of the inertial sensor gyroscope, an error is included in the output of the gesture recognition output. The error will accumulate and expand over time, which will cause the rotation angle of the human body to deviate from the true value. Therefore, this paper adopts the differential evolution algorithm to solve the attitude quaternion, thereby reducing the angle error of the sensor. Secondly, a two-level neural network intelligent motion gesture recognition algorithm is proposed. The two-level neural network intelligent recognition algorithm effectively recognizes similar actions by splitting the traditional single-level neural network into two-level neural networks. The experimental results show that the algorithm in this paper has achieved better recognition performance.
2. Related Research Content
As a branch of the field of pattern recognition, human gesture recognition has received extensive attention and development in recent years [15, 16]. Nowadays, many universities, scientific research institutions, and related companies all over the world have carried out a lot of research on the subject of human body gesture recognition. At present, human motion recognition mainly includes four aspects: vision-based, sensor-based, wireless WIFI signal, and human motion energy-based motion recognition methods [17].
The traditional vision-based motion recognition technology not only requires a camera device to capture the motion trajectory of the human body but also has harsh requirements for the shooting environment. At the same time, the observation angle also has high requirements. Once the observation angle is changed, the recognition of the human body’s actions will also be extremely difficult. Therefore, Liu [18] proposed a new representation method that uses contour information captured by multiple orthogonal cameras to eliminate the impact of viewing angle changes, thereby solving the problem that the viewing angle changes cannot recognize human movements.
In addition to the strict requirements for the shooting environment based on visual recognition technology, the data it collects is also very large. The processing of such enlarged data requires relatively high requirements for the equipment, so the maintenance of the equipment usually requires a lot of money. Due to the unique advantages of sensors, sensor-based identification technology can be applied in more fields. Common sensors include acceleration sensors and GPS sensors [19–21]. Barshan and Yurtman [22] used an inertial sensor to detect the zero speed of the human body during walking and classified and recognized a variety of human daily movements. Kurban and Yıldırım [23] wear a wearable sensor on the wrist and collect acceleration data of the arm for motion recognition.
When the human body moves or undergoes some physical behaviours, the entire body or partial limbs will inevitably produce acceleration. Therefore, the motion state of the human body can be intuitively analyzed through acceleration information. Panwar et al. [24] divided the acceleration data of the human body into six states according to the characteristics of acceleration. Rawassizadeh and Kotz [25] converted the collected acceleration data during human motion into energy data that can be collected through a mass-spring-damping system model and established a public acceleration collection library. The acceleration of the human body in the natural state of motion for more than 200 h can be collected without any restriction. The experimental results show that different energy signals are transformed from different actions of the human body, which provides a new research direction for action recognition based on human movement energy. Ren et al. [26] used a three-axis acceleration sensor to develop a system that can recognize human movement. The system can be used for physical rehabilitation training, medical aid diagnosis, physical exercise, etc.
In recent years, with the continuous development of sensor technology, the size of the sensor is getting smaller and smaller, and the accuracy is getting higher and higher. With the increasing maturity of microelectronics technology, human body gesture recognition based on inertial sensors has gradually become a research hotspot [27]. On the basis of pattern recognition, many researchers apply image-based recognition technology to the field of human body gesture recognition based on wearable devices. In order to solve this problem, Cao et al. [28] used inertial sensor equipment to collect hand movement data and then realized that the recognition of hand movements realizes the efficient interaction between the doctor and the computer. Dong et al. [29] used an acceleration sensor and a bending sensor to complete a wearable wireless sensor device, which realized the function of capturing human arm motion, and completed the capture and recognition of human posture and applied it to medical people. Zeng et al. [30] used inertial sensor equipment to detect the force on the knee joint during jumping and applied it to the cruciate ligament damage detection and recognition, thereby reducing the possibility of human knee joint damage. Guignard et al. [31] used wearable sensors to collect the posture information of volunteers during swimming, assess the correctness of their posture actions during swimming, and provide corresponding references for their later training.
Shinde and Sonavane [32] used wireless body area network technology to study human body posture and constructed a multilevel recognition algorithm for human body posture. A three-layer recognition algorithm was used to realize the recognition of 5 human postures, and the recognition rate reached 95.6%. Fan [33] used an acceleration sensor to conduct experiments on the daily movement of the human body. By acquiring three-axis acceleration data during the movement, the four stepping postures in daily life were distinguished, and the conversion between various postures was carried out to analyze and identify the special situation of a fall. Wan et al. [34] used a three-axis gyroscope and acceleration inertial sensor, applied to software development, and proposed a human body gesture recognition method based on SVM. Truong et al. [35] analyzed the human body’s arm swing walking, push-ups, and sit-ups and extracted the body posture characteristics during the action. The accuracy of step counting is greater than 87.7%; the accuracy of push-ups and sit-ups is greater than 86.3%; the accuracy of heart rate is greater than 90%.
At present, there are still many shortcomings in the recognition of human motion posture, and there is a lack of relevant research in the recognition of motion posture. In response to this situation, this paper proposes an inertial sensor-based motion posture recognition algorithm. This method is more comprehensive and accurate in the recognition and analysis of motion postures.
3. High-Speed Real-Time Augmented Reality Tracking Algorithm Based on Mixed Feature Points
3.1. Sensor Signal Acquisition
With the continuous development and subdivision of the field of pattern recognition, the continuous intelligent and miniaturization of sensors, the recognition of human motion gestures based on acceleration sensors has become more popular. Many researchers have used image recognition technology and speech recognition technology and other related recognition technologies. It has been improved and applied to the recognition of human motion gestures based on inertial sensors.
Figure 1 shows the process of human posture recognition method based on inertial sensors, which is mainly divided into five stages: data collection, data preprocessing, data segmentation, feature extraction, and classifier training. Among them, the data acquisition stage mainly collects physical or physiological signals of the human body through sensor devices, such as acceleration, angular velocity, heart rate, body temperature, and other information. In the data preprocessing stage, it mainly completes the processing of de-drying and normalization of the data, so that the data meets the needs of the system. In the data segmentation stage, the data extraction of a single action in the time domain and the frequency domain space is completed and analyzed separately. The feature extraction stage mainly completes the analysis of unit actions and calculates and extracts relevant attribute features as sample data. Finally, in the classifier stage, the collected samples are constructed according to different classification principles to construct a classification model, and then, the unknown samples are divided.
3.2. Attitude Calculation Algorithm
In order to achieve a more accurate attitude calculation and reduce the noise interference of the sensor, in the process of calculating the node attitude, the three data of angular velocity, acceleration, and magnetic field strength are fused, the space attitude is expressed by the quaternion method, and the differential evolution algorithm is selected to improve the accuracy of the attitude calculation results.
Quaternion is a mathematical concept created by Hamilton in 1843. It is a simple super complex number with the ability to describe the rotation of a rigid body. The human body motion posture studied in this paper is a kind of rigid body rotation. So use quaternion to describe.
Quaternion numbers are composed of a combination of a real number and three imaginary number units, and the definition form is shown in (1).
Among them, , , , and are real numbers; , , and are three imaginary units; and quaternions can also be expressed in the form of . is the quaternion number.
If satisfies , then is called a unit quaternion, and the unit quaternion [1, 0, 0,0] can be used to describe the body’s posture at rest.
Only the unitized quaternion can be used to describe the rotation of the human body, so the quaternion needs to be normalized, and its normalized form is shown in equation (2).
Among them, represents the normalized quaternion.
The unit quaternion can be used to describe the rotation of the human body in three-dimensional space, and its form is shown in equation (3), which is also a differential equation of quaternion.
The formula represents the quaternion, which is composed of the angular velocity detected by the gyroscope, as shown in formula (4).
According to the calculation of complex number arithmetic, there is , and formula (1) and formula (4) are inserted into formula (3) to obtain formula (5).
Equation (5) reflects the relationship between the angular velocity of the human body carrier and the derivative of the quaternion with respect to time. By integrating the derivative, according to the quaternion of the original state, the quaternion in the new state can be obtained, and the quaternion in the new state can be obtained through normalization. The unit quaternion used to describe the transformation of the human body from one posture to the next posture can be obtained, and the update equation is shown in equation (6).
Among them, is a nonnegative integer and represents the time of the system; then, and represent the unit quaternion of the human body posture at the -th and moments, respectively, and represents the time interval between two samples. The value is small, so it is considered that the human body rotates at a constant speed in time.
Since represents the derivative of the quaternion with respect to time at the -th moment, formula (7) can be obtained according to formula (3).
And formula (7) into formula (6) to obtain formula (8).
Therefore, when the initial state of the system and the rotation angular velocity of the human body are known, the quaternion of the state of the system can be obtained from equation (8) to determine the current posture of the human body.
Correspondingly, considering the influence of various disturbances and model residuals, the above evolution equation (8) can be extended to:
In the formula, represents the comprehensive influence of model error and random disturbance on attitude.
3.3. Optimization of Differential Evolution Algorithm
Differential evolution algorithm (DE) is an evolutionary evolution algorithm based on real number coding. Its overall structure is similar to genetic algorithm. The main difference from the genetic algorithm is the mutation operation. The mutation operation of the differential evolution algorithm is based on the difference vector of the chromosome, and the other operations are similar to the genetic algorithm. We use the differential evolution algorithm to quickly obtain the optimal value. Compared with other algorithms for solving nonlinear equations, the differential evolution algorithm has a good calculation effect, is simple and feasible, and has high accuracy.
Let be the -th chromosome of the -th generation, then
Among them, is the length of the chromosome, that is, the number of variables, and is the largest evolutionary algebra. (1)Generate initial population
Randomly generate chromosomes that meet the constraints in the -dimensional space and implement the following measures:
Among them, and are the upper and lower bounds of the -th variable, and is a random decimal between [0, 1]. (2)Mutation operation
Randomly select 3 chromosomes , , and from the population, then: (3)Cross operation
The crossover operation is to increase the diversity of the group. The specific operation is as follows:
Among them, is the crossover probability, . (4)Select operation
In order to determine whether is a member of the next generation; the vector and the target vector are compared with the evaluation function:
Repeat operations (2) to (4) until the maximum evolutionary algebra is reached. The optimal value at this time is the final solution.
3.4. Intelligent Recognition Algorithm of Two-Level Convolutional Neural Network
Figure 2 shows the structure of a two-stage convolutional neural network, which consists of two parts.
The first part is the data collection and segmentation part. Sampling is performed at a sampling rate of 20 Hz. The time length of a sample is 3.2 seconds, which covers a sufficient period of motion posture, thereby improving the stability of the system. And it will not introduce too many cycles of other motion poses when changing motion poses. The 50% overlap rate makes the identification delay around 1.6 seconds, which weighs the impact of system delay and system power consumption.
The second part consists of two cascaded convolutional neural networks. They all have only 2 convolutional layers, 2 pooling layers, and 1 fully connected layer. The first-level convolutional neural network focuses on the classification of motion poses with similar patterns and higher complexity. The second-level convolutional neural network focuses on classifying the four sports postures: walking, jogging, standing, and sitting. Since the types of motion gestures that each level of convolutional neural networks need to recognize are less than those in the single-level end-to-end convolutional neural network intelligent recognition algorithm, they are less complex, so they are all more than single-level end-to-end convolution. The number of parameters of the neural network intelligent recognition algorithm is less.
In common end-to-end neural network algorithms, all motion gestures are directly recognized using a larger single-stage neural network similar in structure to the single-stage end-to-end convolutional neural network intelligent recognition algorithm. The multistage end-to-end convolutional neural network algorithm in this paper uses a separate convolutional nerve to recognize motion gestures. Therefore, it can achieve higher accuracy than a single-stage end-to-end convolutional neural network intelligent recognition algorithm that uses a single-stage convolutional neural network to recognize motion gestures.
Among the sports postures identified in this paper, jogging and fast running have similar postures. In order to solve this problem, this article uses a multistage end-to-end convolutional neural network algorithm. Since the first-level convolutional neural network has roughly recognized these two motion gestures, the second-level network can further recognize more detailed features, so as to achieve accurate classification.
4. Results and Discussion
4.1. Movement Status Division
The degree of dispersion represents the degree of difference between the values of the observed variables, and the difference between the sensor signal sample values is defined as the degree of dispersion. The data dispersion degree of each sensor is calculated, and each movement state can be identified by the threshold value. The principle of the data division method based on dispersion is shown in Figure 3.
(a) Angular velocity dispersion and motion state during walking
(b) Leg angular velocity during walking
(c) Angular velocity dispersion and motion state during passing
(d) Arm angular velocity during passing
Subgraph a is the curve of leg angular velocity dispersion and movement state during walking, and subgraph c is the curve of arm angular velocity dispersion and movement state during passing. In subpictures a and c, the abscissa represents time, and the ordinate represents angular velocity dispersion and action state, respectively. In subfigure a, the athlete starts walking at t1 and stops at t2, during which the legs are in motion. In subfigure c, from time t1 to time t2, the athlete passes the ball, and his arm is in motion. According to the division of the original motion data at t1 and t2 in the two pictures, the angular velocity data in the motion state is extracted, and the subpictures b and d are obtained.
Subgraph b is the leg angular velocity curve during walking, and subgraph d is the forearm angular velocity curve during passing. In subpictures b and d, the abscissa represents time, and the ordinate represents the angular velocity of the arm and leg, respectively. It can be seen from subfigure b that walking is a continuous action composed of multiple unit actions, and the angular velocity of the legs changes periodically during walking. In subfigure d, the angular velocity of the arm does not change periodically during the pass, and the pass is an instantaneous action. It can be seen that the division of the action state can realize the extraction of the movement state data.
4.2. Unit Action Division
The instantaneous action and the continuous action are obtained through the action state division, and the unit action division is the further processing of the continuous action. In the process of continuous movement, it is found through observation that the movement of the legs and arms is in continuous periodic changes, and the periodicity of the continuous changes is more obvious. Therefore, it is feasible to realize the division of unit movements based on the movement data of the arms and legs. It is found that the angular velocity data is the most intuitive in describing the angle change during the human body movement. Therefore, the angular velocity is used as a reference for data division.
During exercise, the sensor signal is easily affected by the human body and the external environment. When calculating the angle of the limb movement, the differential evolution algorithm is used to fuse the acceleration, magnetic field strength, and angular velocity data, which can reduce the influence of external noise. Figure 4 shows the curve of the angle of the calf during walking. In the figure, the abscissa represents time, and the ordinate represents the angle of the calf. The angle curve obtained by the differential evolution algorithm, which changes periodically, but after a period of time, the angle value has obvious deviation; the angle curve obtained by the differential evolution algorithm, basically at 0 degrees both sides fluctuate with equal amplitude. Therefore, the use of differential evolution algorithm to process data can reduce the interference of noise signals.
Figure 5 is a comparison diagram of angular velocity and angle during walking and dribbling. In the figure, the abscissa represents time, the ordinates of subgraphs a and c represent the angular velocities of the forearm and calf, respectively, and the ordinates of subgraphs b and d represent the small. It can be seen from subgraphs a and c that there are more noise signals in the angular velocity signal, and the curve is not smooth enough; while the angle signal curves in subgraphs b and d are relatively smooth, so the unit action can be divided based on the angle to reduce implementation complexity.
(a) Forearm angular velocity when dribbling
(b) Forearm angle when dribbling
(c) Calf angular velocity when walking
(d) Calf angle when walking
4.3. Attitude Calculation Accuracy Test
In this paper, the differential evolution algorithm is used to calculate the quaternion of the human posture in order to filter the system error caused by the angle drift of the gyroscope. Therefore, it is necessary to determine the accuracy of the posture calculation of the sensor. In the experiment, the method of reading the rotation angle of the sensor node obtained by differential evolution with the actual angle is used to determine the accuracy of the attitude calculation under the condition of determining the rotation angle.
The experiment starts at an angle of 0 degrees and continues to rotate the sensor node at the end of an angle of 1080 degrees, sampling data every 90 degrees. The experiment is divided into two groups. In order to compare the compensation effect of the differential evolution algorithm for angle calculation, the first group of the experiment does not use any compensation method and directly obtains the sensor rotation angle through angular velocity integration; the second group uses the differential evolution method to compensate the angle output of the sensor node.
Figure 6 shows the comparison of the error data before and after compensation. After rotating by 1080 degrees, the angle calculation error without any compensation method has reached 10 degrees, while the angle calculation error compensated by the differential evolution algorithm is only 0.37 degrees. It can be seen that the differential evolution algorithm can effectively compensate for the angular velocity integral to improve the accuracy of human body posture calculation.
The completion of the movement is mainly through the overall movement of the athlete’s upper and lower limbs coordinated movement, so when the movement is recognized, it is necessary to discuss the movement of the upper and lower limbs. Through the combination of upper and lower extremity movements, the movements of the athletes are determined. This paper compares the classification performance of the proposed two-level convolutional network and different classifiers for gesture recognition. As shown in Figure 7, the average recognition rate is up to 99%, and the average recall rate is up to 99%. Experiments show that the two-stage convolutional network proposed in this paper has the best effect on the recognition of upper and lower limb movements.
(a) Different classification algorithms to catch the ball action classification
(b) Different classification algorithms to pass the ball action classification
(c) Different classification algorithms to dribble the ball action classification
(d) Different classification algorithms to shoot the ball action classification
During the training process, the error curve performance of the two-level convolutional network, BP algorithm, SVM algorithm, BN algorithm, and C4.5 algorithms is shown in Figure 8. Through comparison, it can be found that after 500 iterations of the training part, the iteration error value of the two-stage convolutional network is the smallest, reaching the set accuracy. The BP neural network algorithm has the largest iterative error value, which is much different from the set accuracy. At the same time, the curve of SVM algorithm and BN algorithm declines slowly, and after more iterations, a smaller error value is reached.
We further compare the algorithm in this paper with the most advanced methods at present, and the experimental results are shown in Table 1. It can be seen that the algorithm in this paper has achieved the highest recognition accuracy.
5. Conclusion
Human body gesture recognition has attracted more and more attention in various fields, such as medical treatment, sports, games, and movies. In this paper, the sports posture recognition of athletes in the sports field is researched and analyzed. The motion state information of the human arms and legs is detected by sensor equipment to complete the recognition of the sports posture. In this paper, inertial sensor technology is applied to attitude recognition in motion. First, in order to improve the accuracy of attitude calculation and reduce the noise interference in the preparation process, this article uses differential evolution algorithm to apply attitude calculation to realize multisensor data fusion. Secondly, a two-level neural network intelligent motion gesture recognition algorithm is proposed. The two-level neural network intelligent recognition algorithm effectively recognizes similar actions by splitting the traditional single-level neural network into two-level neural networks. Experiments show that the experimental method designed in this article for the posture in motion can obtain the motion information of the examinee in real time, realize the accurate extraction of individual motion data, and complete the recognition of the motion posture.
Although the algorithm in this paper has achieved high accuracy, the real-time performance needs to be further improved. Our future work is to improve real-time performance while maintaining accuracy.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This study was supported by the Major Project of Basic Research on Philosophy and Social Sciences in Henan Province Colleges and Universities: The Power of Sports: Research on the Diversified Value and Function of Sports from the Perspective of Socialist Core Values (2021-JCZD-34). This work was supported in part by the National Natural Science Foundation of China under Grant U1804152.