Abstract
Correctly identifying human activities is very significant in modern life. Almost all feature extraction methods are based directly on acceleration and angular velocity. However, we found that some activities have no difference in acceleration and angular velocity. Therefore, we believe that for these activities, any feature extraction method based on acceleration and angular velocity is difficult to achieve good results. After analyzing the difference of these indistinguishable movements, we propose several new features to improve accuracy of recognition. We compare the traditional features and our custom features. In addition, we examined whether the time-domain features and frequency-domain features based on acceleration and angular velocity are different. The results show that (1) our custom features significantly improve the precision of the activities that have no difference in acceleration and angular velocity; and (2) the combination of time-domain features and frequency-domain features does not significantly improve the recognition of different activities.
1. Introduction
The classification of human motion based on inertial sensors has been proven to have many important applications in the medical and health fields. In previous studies, time-domain and frequency-domain features are widely used for feature calculation.
There are many studies that use wavelet transform to extract features to classify human activities. However, the research of Preece et al. [1] shows that the time- and frequency-domain features often exceed wavelet features, indicating that the wavelet feature may be not the most effective method for calculating the human body motion classification features.
Some time-domain features are derived to classify human activities, such as the mean, median, variance, skewness, kurtosis [2], and interquartile range [3]. In order to extract frequency-domain features, the sensor data window is first changed to the frequency domain using discrete Fourier Transform [4]. Then, we can extract some features from the frequency domain to distinguish different activities, such as power spectral density (PSD) [5], peak frequency [5, 6], entropy [7], DC component [7], median frequency [8], spectral energy [9], and frequency-domain entropy [10]. Of course, there are other methods that process data from accelerometers and gyroscopes. But all in all, to the best of our knowledge, these features are extracted directly from acceleration and angular velocity, which inevitably have some common drawbacks.
We studied 12 kinds of activities and found it easy to confuse elevator up and elevator down. These two kinds of activities do not have obvious differences in acceleration and angular velocity, so the time and frequency-domain features based on acceleration and angular velocity cannot achieve good classification results. Therefore, for those motions that have no significant difference in angular velocity and acceleration, no matter how the features are extracted from the acceleration and angular velocity, it is difficult to achieve good results.
In addition, we begin to wonder if there is any essential difference between time-domain features and frequency-domain features based on acceleration and angular velocity. In order to solve this discredit, we have separately tested the effects of time-domain features and frequency-domain features. Then, we tested the combination of the two kinds of features and found that the combination of time-domain features and frequency-domain features was slightly higher than only time-domain features. From the experimental results analysis, we believe that, for human activity classification problem, the time-domain features and the frequency-domain features are two aspects of the same rules, and there is no essential difference.
Our contributions in this paper are two-fold: (1) to the best of our knowledge, this is the first time that features of motion classification based on velocity and displacement have been proposed, which solve some problems that cannot be solved by traditional time- and frequency-domain features; (2) As far as we know, for the first time, we have studied the difference between the time-domain features and frequency-domain features.
2. Methods
Indeed, the time-domain and frequency-domain features based on acceleration and angular velocity have achieved some success. However, for activities without obvious difference between acceleration and angular velocity, such as elevator up and elevator down, the traditional method of extracting features based on acceleration and angular velocity is difficult to work. In order to solve this problem, we carefully analyze the two activities of elevator up and elevator down, summarize the differences between them, and propose some new features for distinguishing such activities. After analysis, we sum up the following rules:(i)When the elevator goes up, the speed is upward; when the elevator goes down, the speed is downward.(ii)When the elevator just starts to move or stops moving, its speed is small and its angular speed is large.(iii)When the elevator just starts to rise or fall, the direction of the speed is the same as the direction of the acceleration; when the elevator stops to rise or fall, the direction of the speed is opposite to the direction of the acceleration.
Based on the above evidences, we propose four features to distinguish these activities on each axis of the accelerometer. First, according to the first rule, we introduce three features, namely, the starting speed, the ending speed, and the displacement. Second, according to the second and third rules, we introduce the fourth feature. As some activities have just started and are about to stop, their speed is small and difficult to distinguish. Therefore, we extracted another feature to enhance the difference between two activities. When the velocity direction is the same as the acceleration direction, we use as the feature; otherwise, we use as feature. In order to describe the movement of the human body in different directions as much as possible, we have introduced the following twelve new features to enhance the difference between different activities. These features are summarized in Table 1.
Suppose the time window we choose is T, the sampling frequency is n, then the total number of samples is Tn. In the experiment, the time window we selected was two seconds. The displacement, time, and acceleration corresponding to the ith sampling interval are x(i), t(i), and a(i). The speed corresponding to the sampling point is . Data section of one time window is shown in Figure 1. The gravity acceleration is . Due to the high sampling frequency, the time interval between each sample point is short. At the same time, in order to simplify the calculation, we believe that there is uniform linear motion between each sampling point. Now, we derive the speed and displacement formula.
We think these sampling points are equally time-distributed, so we have the following conclusions.
First, we derive the formula for the end speed of each axis. According to the kinematics formula, we can easily get the following formula and simplify it by combining it with Equation (1).
Next, we introduce the determination of the starting speed. If this window is the first window, we default to a starting speed of zero. Otherwise, the starting speed is the end speed of the previous window marked as :
As for the displacement of each axis, since we think that there is uniform linear motion between each sampling point, the displacement formula can be derived as follows.
In the experiment, in order to calculate the last three features we defined in Table 1, that is, Velocity Plus Acceleration Along X, Velocity Plus Acceleration Along Y, and Velocity Plus Acceleration Along Z, our speed takes the end velocity of the two-second time window, and the acceleration takes the difference between the average acceleration of the two-second time window and the gravitational acceleration in each axis. In this way, we can calculate the last three features, and the expression is shown in (6), in which Velocity Plus Acceleration Along∗ stands for the last three features.
Finally, we introduce the calculation method of gravity acceleration we used in the experiment. Since the tester is just wearing the device for data acquisition, it is generally at a standstill and the starting speed is zero, which is used in our experiments. Therefore, for the sake of convenience, in the experiment, we believe that the initial acceleration of each axis is the component of gravity acceleration and assume that the component of gravity acceleration in each axis remains unchanged. We take the average of the first 10 sampling points of each axis as the component of gravity acceleration in each axis, recorded as .
3. Experiments and Results
3.1. Datasets
In order to illustrate the validity of our custom features, we have selected the USC_HAD of University of Southern California as the verification dataset [11]. They use an off-the-shelf sensing platform called MotionNode to capture human activity signals and build their dataset. MotionNode is a 6-DOF inertial measurement unit (IMU) specifically designed for human motion sensing applications, which integrates a 3-axis accelerometer and, 3-axis gyroscope. They selected 14 subjects (7 male; 7 female) to participate in the data collection. The sampling frequency is 100 Hz. Twelve kinds of activities collected are Walking Forward, Walking Left, Walking Right, Walking Upstairs, Walking Downstairs, Running Forward, Jumping Up, Sitting, Standing, Sleeping, Elevator Up, and Elevator Down.
3.2. Results
In order to verify the validity of our custom features, we extracted some common time- and frequency-domain features. In the time domain, we chose the mean, median, variance, skewness, kurtosis, and interquartile range as time-domain features. In the frequency domain, we choose peak frequency, median frequency, power spectral density, DC component, spectral energy, and information entropy as frequency-domain features. Then, we added our custom features to these time- and frequency-domain features and compared their results. In the experiments, we adopted the two commonly used models, SVM and random forest. For SVM, the kernel function we use is a polynomial kernel function. For RF, the number of decision trees we choose is 50.
First, in order to check whether the distinction between the lift of the elevator and the descending of the elevator is achieved, we tested the precision and recall of our custom features on both models. Precision and recall are often used as performance measures for classifiers in classification problems. Table 2 shows the precision and recall of the models’ identification of elevators up when adding custom features to time- and frequency-domain features and the combination of time and frequency-domain features without custom features. Table 3 shows the precision and recall of the models’ identification of elevators down when adding custom features and no custom features. The left column below each model is the precision rate, and the right column is the recall rate. From Tables 2 and 3, we can see that, after adding the custom features, the model has significantly improved the recognition rate of the elevator up and the elevator down.
Also, we use the ROC (receiver operating characteristics) curve and corresponding AUC (area under the ROC curve) values to check whether the distinction between the lift of the elevator, and the descending of the elevator is achieved. For SVM, we conducted a test. The ROC curves for the two types of classification results for elevator up and elevator down are shown in Figure 2.
From the above figure, we can see that in the SVM, after adding our custom features, the ROC curves of elevator up and elevator down completely cover the ROC curve of the original feature. The AUC value for the no custom features’ ROC curve is 0.8572, while the AUC value for adding custom features’ ROC curve is 0.9729. The ROC curve shows that our custom features have achieved very good results in distinguishing between the elevator up and the elevator down.
In order to further explain the significance of our custom features we have extracted and verify the difference between time-domain features and frequency-domain features, we have compared our custom features with the time-domain features and frequency-domain features. We performed comparative experiments on four combinations of features over SVM. We conducted five experiments for each experiment. The detailed experimental results are summarized in Table 4. The accuracy in the table is the total classification accuracy of the 12 activities. For convenience, we denote the time-domain feature as , the frequency-domain feature as , and the custom feature as .
From the experimental results, we can see that individual frequency-domain features and time-domain features can achieve good results. However, when frequency-domain features are combined with time-domain features, no significant improvement is obtained, indicating that there is no essential difference between features extracted from the frequency domain and features extracted from the time domain. The superposition of the two did not achieve better results. The time-domain feature is better when combined with our custom features than combined with frequency-domain features. Based on the comprehensive analysis, we believe that our custom feature is a supplement to traditional time-domain features rather than a redundant feature.
The traditional time-domain features and frequency-domain features are all based on the acceleration and angular velocity but there is no essential difference, so the superposition of the two will not bring significant improvement. Our custom feature is the mining of the rules of speed and displacement, which is very different from the traditional mining of acceleration and angular velocity. So, when these features are introduced in the time-domain feature, we can obtain certain promotion. Especially for those motions which have no obvious difference between angular velocity and acceleration but there is a clear difference in speed and displacement, we can achieve good results with these custom features. For example, there is no obvious difference in acceleration and angular velocity in the smooth upward movement of the elevator and the smooth descending of the elevator, but there is a clear difference in speed and displacement. So, when introduce our custom feature, we can obviously increase the recognition rate of the two kinds of motions.
To calculate these features we define, we must know the initial state of motion, especially the initial state of speed. In our experiment, we assumed that the initial state is zero. In the database we use, most of the data are collected after the tester reaches a steady state of various motion postures, which does not satisfy our assumptions. If we can record data when the tester just wears the inertial sensors, so as to meet our assumptions, we believe we can achieve better results. For other types of sports, such as running, walking, and station, there is a difference in speed, which will bring a higher recognition rate.
4. Discussion
In this article, we begin with the elevator up and elevator down, which are indistinguishable based on the existing feature extraction methods and analyze the differences and rules between these two types of movements. Then, we have proposed four features on each axis of the accelerometer that have significantly improved the distinction between the two types of movements, elevator up, and elevator down. In the experiment, we found that the combination of frequency-domain features and time-domain features does not significantly improve the distinction of activities. The two kinds of features are two different aspects of acceleration and angular velocity, and there is no essential difference. From the experimental results, the time-domain features are better than the frequency-domain features and can more fully reflect the differences between different activities. Our custom features are not another response to acceleration, but instead, these features can be used to distinguish movements that differ in the speed of movement. In particular, it is of great significance to distinguish between movements that do not have a significant difference in acceleration and angular velocity but have a significant difference in speed.
In the experiment, for the sake of convenience, we assumed that the component of the gravitational acceleration remains unchanged, which is obviously not in line with the actual situation. Next, related personnel may consider introducing some basic theories of motion analysis in order to accurately calculate the components of the gravitational acceleration and thus more accurately calculate the features we introduce. We believe that when the features of velocity and displacement are introduced, we can make a great breakthrough in the existing human motion classification problem and to some extent get rid of the dilemma that some motions cannot be accurately identified on the features of acceleration and angular velocity.
Data Availability
The USC_HAD data used to support the findings of this study are included in Reference [11].
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported by the Natural Science Foundation of China (No. 61571268).