Walking Gait Phase Detection Based on Acceleration Signals Using Voting-Weighted Integrated Neural Network

Yan, Lei; Zhen, Tao; Kong, Jian-Lei; Wang, Lian-Ming; Zhou, Xiao-Lei

doi:https://doi.org/10.1155/2020/4760297

Complexity

On this page

Abstract Introduction Materials and Methods Results and Discussion Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Special Issue

Dynamical Analysis of Biological Systems

View this Special Issue

Research Article | Open Access

Volume 2020 | Article ID 4760297 | https://doi.org/10.1155/2020/4760297

Walking Gait Phase Detection Based on Acceleration Signals Using Voting-Weighted Integrated Neural Network

Lei Yan,¹Tao Zhen,¹Jian-Lei Kong,²Lian-Ming Wang,¹and Xiao-Lei Zhou¹

Guest Editor: Eberhard O. Voit

Received12 Sept 2019

Revised20 Nov 2019

Accepted12 Dec 2019

Published08 Jan 2020

Abstract

Human gait phase recognition is a significant technology for rehabilitation training robot, human disease diagnosis, artificial prosthesis, and so on. The efficient design of the recognition method for gait information is the key issue in the current gait phase division and eigenvalues extraction research. In this paper, a novel voting-weighted integrated neural network (VWI-DNN) is proposed to detect different gait phases from multidimensional acceleration signals. More specifically, it first employs a gait information acquisition system to collect different IMU sensors data fixed on the human lower limb. Then, with dimensionality reduction and four-phase division preprocessing, key features are selected and merged as unified vectors to learn common and domain knowledge in time domain. Next, multiple refined DNNs are transferred to design a multistream integrated neural network, which utilizes the mixture-granularity information to exploit high-dimensional feature representative. Finally, a voting-weighted function is developed to fuse different submodels as a unified representation for distinguishing small discrepancy among different gait phases. The end-to-end implementation of the VWI-DNN model is fine-tuned by the loss optimization of gradient back-propagation. Experimental results demonstrate the outperforming performance of the proposed method with higher classification accuracy compared with the other methods, of which classification accuracy and macro-F1 is up to 99.5%. More discussions are provided to indicate the potential applications in combination with other works.

1. Introduction

As the most common form of human behavior, walking style is related to health status and individual differences, which can be shown by the differences of the gait phase [1]. Detecting results of the gait phase can provide references for disease diagnosis and rehabilitation [2, 3], which is of great significance to the patients’ clinical rehabilitation. For example, an estimated gait disorder of 1.1 million children may have originated from different somatosensory disease in the United States [3]. In addition, researchers have managed to program humanoid robots to use human-based gait trajectories generated via gait classification [4], as well as consistently control wearable assistive devices such as robotic prostheses [5] and orthoses [6] for the recovery of lower-limb mobility. For instance, Yan et al. [4] proposed that gait phase detection can also be used to facilitate the development human auxiliary equipment, such as the medical ankle joint (AF), hip joint (HK), and knee ankle joint (KAF) orthopedic devices, as well as exoskeletons and other equipment. Similarly, gait phase detection plays an important role in sports medicine [7] and rehabilitation medicine [8].

Computational methods for gait phase recognition fall into two main categories. The first category is comprised of algorithms, which divide the gait phases based on the threshold selection of either raw or processed data [9]. Secondly, some deep-learning approaches have emerged in recent years to substitute the aforementioned techniques that rely on traditional classification algorithms. Some have applied deep-learning algorithms to different types of sensors to detect gait phases. For instance, Mukherjee et al. [10] present a fully automated frontal (i.e., employing front and back views only) gait phase recognition approach using the depth information captured by multiple Kinect RGB-D cameras. However, the captured image information is easily disturbed by the external environment. Rosati et al. [11] proposed a method of hierarchical clustering to achieve recognition of the human gait phase by processing electromyography (EMG) data collected during gait, which has improved the abovementioned problems that are susceptible to environmental interference. However, muscle electrical signals are susceptible to factors such as sweat when collecting EMG data. Ding et al. [12] further improved the problems of the above EMG method and proposed a proportional fuzzy algorithm to achieve smooth recognition of the gait phase for foot pressure information processing, but the foot pressure will be affected by the wearer’s weight, load, and other factors [13], and the pressure sensor also has a high failure rate. In recent years, researchers have started to study gait phase recognition methods based on inertial sensors (IMU). This is mainly due to the fact that more information can be obtained by adopting a small number of inertial sensor modules and most of the inertial sensor modules are installed on the legs and feet, so as to avoid damage or discomfort to the wearer [14]. At the same time, the information of IMU is basically unaffected by human body weight, belongings, clothing, sweat, and other factors, which is a prominent advantage compared to the method of plantar pressure or muscle electrical signal detection. In addition, inertial sensors are extremely cost effective [15] and acceleration signals acquired by inertial sensors exhibit typical waveform characteristics during the gait cycle. Previous studies have positioned inertial sensors on the instep, thigh, and calf [16–18]. This paper considers the position of the instep, lower leg, and thigh because the classifier has better classification performance at the lower extremity position [19].

For the recognition system, this paper designs an effective and adaptable gait detection method. Some research studies [13] indicate that a large amount of information can be obtained by using a small number of acceleration sensors that are located on the legs and feet to minimize sensor damage and discomfort to the person wearing the sensor. In this paper, we describe a system that uses three inertial sensor modules to obtain the acceleration information of the lower limbs of the human body. The collected acceleration data was reduced by the Principal Component Analysis (PCA) algorithm, which focuses on extracting the feature information of the original data and searches for a set of orthogonal low-wiki functions to represent a set of high-dimensional data, improving the recognition rate and recognition speed [20, 21]. Then, the paper divides the human gait into three phases and proposes a method of dividing the three gait phases. Finally, this paper proposes a VWF-DNN algorithm for detecting the gait phase, which is inspired by integrated learning. The core idea of the VWF-DNN algorithm is to use the three subneural networks with distinct differences to output the final classification result through the voting algorithm designed in this paper. The designed VWF-DNN with higher accuracy will be further evaluated with learned and unlearned data to test its suitability with acceleration classification.

This paper proposes an algorithmic model for detecting the gait phase, which uses the acceleration data from the instep, calf, and thigh to accurately detect two gait phase events. Finally, the effectiveness of the proposed VWI-DNN algorithm in gait phase detection is verified by the final recognition results.

2. Materials and Methods

2.1. Data Collection

Twenty volunteers with weight range in 46 kg to 88 kg and height range in 155 cm to 190 cm were recruited for the experimental data collection. The details of personal information are shown in Figure 1. The subjects have no physical or nerve injury to their legs or feet, which may affect walking gait phase detection.

With the improvement of the sensor manufacturing process, this study selected three IMU modules as portable devices for obtaining acceleration information. The inertial sensor modules were placed on the foot dorsum, the outer side of the lower leg, and the outer side of the thigh. The arrangement of the acceleration sensor on the instep, calf, and thigh monitoring the lower limb movement is shown in Figure 2. The acceleration resolution of the three-axis inertial sensor module used in the experiment is 6.1e − 5 g, the stability of attitude measurement is 0.01°, and the transmission baud rate in the experiment was set to 115200 bps.

In this experiment, all participants were asked to walk for at least 120 s on the configured treadmill with speeds at 0.78 m/s, 1.0 m/s, and 1.25 m/s, respectively. Participants walked normally three times on a treadmill at each speed, with all settings being the same in each state. In order to prevent the participants from affecting the gait due to fatigue, the experiment requires the participants to rest for 2 minutes for each walking test. In addition, data is only saved until the treadmill’s running speed reaching the set speed. When the experiment was stopped and the treadmill began to slow down, we stopped collecting data. Moreover, each participant was asked to perform the same experiment under the same conditions to ensure the reliability and validity of the collecting process.

2.2. Data Preprocessing

Since each data sample contains multiple features from different sensors and each data in the same IMU module includes three acceleration data in X, Y, and Z directions, abundant data with different dimensions will lead to excessive complexity and easy overfitting of the detection model. In order to reduce the dimension of data set, the PCA method was adopted to synthesize the three-directional acceleration information of every IMU sensors into a new dimension variable Comp. PCA [22] is a general tool for dimensionality reduction and data analysis, and its essence is to project the data samples in the high-dimensional space into the low-dimensional space through linear transformation, while preserving the original data features as much as possible [23]. With the dimension reduction process, the compressed Comp can avoid the excessive information lose and adjust input dimension before passing acceleration data into the subsequent classifier. The Comp is calculated as follows:where , and represent the acceleration in the X-, Y-, and Z-directions, respectively. Comp is a one-dimensional data by combining three-directions movement, which can improve the response performance of the algorithm and avoid overfitting during late training. The combined acceleration “Comp” of the instep, calf, and thigh together constitutes the input vector of the model, where z_1, z_2, and z_3 represent the coefficients of the acceleration in three directions. The distribution of z_1, z_2, and z_3 corresponding to different body parts at asynchronous speed is shown in Table 1.

Based on the above experiment, we can get the curve of the acceleration in the X, Y, and Z directions and the combined acceleration as shown in Figure 3.

(a)

(b)

(c)

The human walking process is a rhythmic movement, and a complete gait cycle definition is from the unilateral heel to the ipsilateral heel again [21]. A two-phase model has proven to be sufficient to control the knee module of an active orthosis [9]. Nonetheless, the most widespread approach relies on a four-phase model [24], which are independently written as (1) Heel Strike (HS), (2) the loading response phase or Flat Foot (FF), (3) the heel lifting or Heel-Off (HO) and (4) the initial Swing Phase (SW). This four-phase model of gait granularity has been used for the actuation of multiple robotic ankle-foot orthoses [25, 26].

To ensuring the scientificity of gait classification, the walking cycle in this paper was also divided into HS, FF, HO, and SW. During normal walking, the acceleration signals in the three directions of the foot, thigh, and calf exhibit periodicity. The sway phase accounts for approximately 40% of the total gait phase, and the stance phase accounts for approximately 60% of the total gait phase [27]. We can approximate that the stance phase is the biggest phase in the walking cycle. According to the division of gait phase, the phase division in this paper is shown in Figure 4.

Except for the gait phase division, feature selection is also used to extract meaningful information or noise from acceleration signals. After this processing, the key features effectively representing different gait phases are obtained from time domain for the subsequent recognition model. In this paper, the standard deviation (SD), mean absolute value (Mav), maximum value (Max), minimum value (Min), and median (Med) are selected to handle with acceleration signals as feature vectors. Since the vectors composed of single and multiple feature sets will produce different accuracy rates, we merged SD, Max, Min, Med, and Mav feature vectors to form the input feature vector in order to improve the recognition accuracy.

2.3. Voting-Weighted Integrated Neural Network

The next step is to design an algorithm to recognize the timings vectors related to the acceleration signal. As the commonly classifier with excellent performance [28], the DNN is a feedforward artificial neural network consisting of an input layer, an output layer, and at least two hidden layers [29]. Although the DNN is a strong classifier, sometimes the subneural network still misjudges certain situations, leading to the misclassification of results [30]. Thus, the voting fusion of the neural network is proposed to solve the instability of the subneural network, which may be insensitive to the data of some input layers due to a single network structure [31], and results in errors in the output layer. The output of the integrated neural network is determined by the output of each integrated neural network under the sample [32], which can improve the classification performance and generalization performance of the classifier to some extent [33, 34]. Therefore, inspired by the AdaBoosting algorithm [35] and Bagging [36] algorithm in current integrated learning, this paper further proposes a novel VWI-DNN algorithm (the entire structure is shown in Figure 5) by modifying the traditional DNN network. Our work aims to construct a general neural network structure allowing for different voting decisions at each submodel and demonstrate its use as a practical way to massively increase model capacity.

Firstly, we chose three advanced deep neural networks instead of the perceptual layer network as the classifiers and named them SNN_1, SNN_2, SNN_3, so as to improve the classification performance of the network. The research mainly focuses on the design of three subneural networks and the optimization of network parameters, as well as the fusion method of the output results of these three subneural networks. SNN_1, SNN_2, and SNN_3 have one common input layer and two hidden layers, so that they can have a common network structure. In addition, three independent and different networks are set up in the third hidden layer. This design can simplify the network and share the weight information. The latter network structure maintains their independence, the number of neurons, and activation functions, and the number of hidden layers exist certain differences. The neural network has associated nonlinearities and are trained using greedy hierarchical supervision, and the final learning rate was manually specified and set to 0.05. Finally, the three subneural networks output their respective classification results through the Softmax regression layer. Some parameters of the structure of the entire network are shown in Table 2, where HL_x represents the xth hidden layer and the “Dropout column” represents the sparse rate that needs to be set.

The goal of the WWI-DNN algorithm is to identify the three gait phases of the human body, which solves the multiclassification task. However, the output of the neural network does not necessarily represent a probability distribution, so the output of the neural network must be transformed into a probability distribution through the Softmax regression layer, whose expression is shown in the following equation:

Then, the focus of the VWI-DNN algorithm is to establish a voting fusion mechanism. In the process of output fusion, the establishment of fusion algorithm is the core content of information fusion [37, 38]. The concept of voting fusion was therefore proposed to obtain more accurate results from multiple unreliable data. After years of development, there have been numerous voting algorithms, such as majority voting fusion, logical voting fusion, median voting fusion, and weighted average voting fusion. The voting weight of an excellent voting fusion algorithm should not be artificially set. It is more reasonable that its voting weight should be larger when a subneural network performs well and vice versa. In the AdaBoosting algorithm theory, the weight information is updated with the classification error rate, but the weight update is problematic when the error rate is higher than 50%. In order to avoid this problem, considering that good performance classifiers should be rewarded and poor performance classifiers should be punished, this paper proposes a weighted sum voting algorithm. The algorithm of weighted summation is to add the weights of the three networks to output the same result, respectively, and regard the sum of the weights as the fusion result. This is also a simple and effective information fusion algorithm.

In order to solve the multiple classifier weight problem, this paper introduces the weighting function. This paper used the classification accuracy to determine the weight coefficient of each classifier. Equation (3) shows how to solve the accuracy of each classifier, and we need to set the corresponding weighting function to determine the weight coefficient of the classification. The image of the weighting function is given in Figure 6, and its expression is as shown in equation (5), wherein the expression of the reward function is indicated in equation (4). It can be seen that the derivative value of the reward function (Deriv_reward) has a negative correlation with the value of the abscissa, which is why this paper chooses it as a reward function. Such a function can make the model with good performance get larger reward. When the accuracy is less than 40%, the weight should be reduced, so the weight coefficient is taken as ; similarly, when the accuracy is more than 40%, the classification performance of the weak classifier can be considered great, and its weight should be increased, so the weight should be taken . Meanwhile, in order to make the new weight available in [0,1], this paper normalizes c by using equation (6). In addition, given that one classifier cannot be made large on its own, this paper sets a minimum threshold of 0.26 for each classifier’s weight. The classification weight of each classifier is obtained according to equation (7). The given initial value is 33.33% and the initial value of is 1.0.

The next step is to confirm the weight of the classifier. The classification result of each classifier is calculated by using equation (8). When the classification results are the same, the weights of these classifiers should be added and obtained the weight corresponding to each gait phase. This process can be expressed by equation (9), where indicates the probability output corresponding to the human gait phase k. The final classification result should be determined by the maximum weight value corresponding to each gait phase, thereby obtaining the final integrated output q, whose expression is as shown in equation (10), where denotes the number of samples correctly classified, denotes the total number of samples, denotes the value output by the ith subneural network through the output layer, and denotes the weight of the ith neural network corresponding to the gait phase k, and when k = 1, it represents the starting phase; when k = 2, it indicates the swing phase; when k = 3, it indicates the foot phase.

Finally, the task of the VWI-DNN algorithm is to solve the problem of internal parameter update and optimization of each subneural network. The neural network generally updates the internal parameters of the network architecture by optimizing the loss function value. When using neural networks for classification, the usual function is the cross-entropy loss function [39] which characterizes the distance between two probability parts so that the cross-entropy loss function can be used to calculate the distance between the predicted probability distribution and the probability distribution of the real answer. According to the cross-entropy loss function equation (11), we can obtain the loss function equations (12)–(14) of the three subneural networks. When we train this network, we hope that the parameters of the first subneural network would not be changed when we train the other two neural networks. But the reality is that when you train any of the three subneural networks, the parameters of the shared layer will change. Therefore, if the optimizer optimizes the three loss functions separately, the shared layer parameters will be changed reciprocally and the ideal result will not be achieved. In order to improve this situation, this paper proposes a method to optimize the global loss by using a classifier. The optimizer no longer optimizes the three cross-entropy loss functions separately but only optimizes the sum of the three loss functions (loss_sum), which can ensure that the shared layer parameters can achieve a better result and the training speed can be improved. The loss_sum expression is shown in equation (15).where denotes the probability distribution of the three subneural networks predicting the phase of the three types of gaits after passing through the Softmax layer and denotes the distribution of the real samples.

To avoid overfitting, 70% of the sample set was selected for training and 30% for testing. After training the three learning models 10,000 times with the same training set, the same test set was used to test the trained models, and the classification accuracy, macro-F value, and area ender curve (AUC) after the test were recorded. The entire process of this study is shown in Figure 7.

3. Results and Discussion

3.1. Evaluation Methods

Comparing the classification performance of different classifiers cannot determine the effectiveness of the algorithm by a single metric and there are many other methods commonly used to obtain a full census. The evaluation metrics including Precision, Recall, Accuracy, and F1-score are used to compare different methods. Precision and Recall are widely used in the fields of information retrieval and statistical classification to evaluate the quality of results, where the higher the Precision and Recall value, the better the method performs. F1 combines the results of P and R, and when F1 is high, it indicates that Precision and Recall are both high, and this evaluation index is relatively effective. However, the classifier of this paper performs multiclassification task. We hope to comprehensively investigate the Precision and Recall on several binary confusion matrices, and the most direct way is to calculate macro-F1 [40]. While, Accuracy reflects the ratio of the samples correctly classified by the classifier to the total samples for a given test data set. According to equations (16)–(21), we can calculate these evaluation factors, where TP, TN, FP, and FN, respectively, represent true positive, true negative, false positive, and false negative.

In order to better analyze the performance of the classifier, this paper introduces the AUC under receiver operating characteristic (ROC) as the evaluation index of the algorithm. Spackman was the first to adopt ROC diagram for machine learning, and he proved the value of the ROC curve in evaluation [41]. In recent years, it has been applied more and more in machine learning and data mining research, partly because people realized that simple classification accuracy is usually not a good indicator to measure performance [42]. The AUC of each algorithm can be calculated to compare, and the algorithm that has the largest AUC will have the best diagnostic value:

3.2. Results

The joint confusion matrix of three gait phase recognition results at different synchronization speeds are, respectively, shown in Figures 8–10. According to Figures 8–10, we can easily get Tables 3–5 which, respectively, classifies the performance for each training function in terms of HS, FF, HO, and SW phase under three kinds of sync speed. According to Tables 3–5, it can be observed that all of Bagging, Boosting, and VWI-DNN have macro-F1 of HS and SW phase recognition nearly up to 100%. While from the results obtained by F1, Bagging, and AdaBoosting have poor recognition effects on FF and HO phases. In particular, Bagging has a case where macro-F1 of the FF and HO phases is 0. It can also be clearly seen from the observation of Figure 8 that the Bagging algorithm can easily recognize the FF phase as the HO phase and the HO phase as the FF phase. The Bagging algorithm is extremely poor in HO phase and FF phase recognition. It can be seen from Table 5 that although Bagging and AdaBoosting have poor recognition of FF phase and HO phase, VWI-DNN algorithm proposed in this paper still has more than 98% macro-F1 value for the FF and HO phase. The three training functions have higher accuracy for SW and HS phases recognition at three paces, generally reaching more than 98%. As aforementioned, macro-F1 can comprehensively measure the two indicators of Precision and Recall. It was observed that macro-F1 value of VWI-DNN is more than 98% for any phase at any pace.

(a)

(b)

(c)

(a)

(b)

(c)

(a)

(b)

(c)

In order to comprehensively measure macro-F1 in four gait phases, macro-F1 was introduced in this paper. According to equations (16)–(20), the corresponding macro-F1 can be calculated, and the corresponding Table 6 was obtained. According to Table 5, the recognition accuracy of the VWI-DNN algorithm is as high as 98% or more, while the other two training functions have lower accuracy. Especially, the Bagging algorithm has a phase recognition accuracy of less than 77% at three paces. It can be clearly seen from Figures 11 and 12 that the VWI-DNN algorithm is higher in accuracy and macro-F1 than the other two algorithms at any of the paces. By observing the AUC, we can also see that the VWI-DNN algorithm can reach 1.0, which is high. Figure 13 also illustrates the relationship between the three algorithms corresponding to the AUC at the unsynchronized paces. From Figure 14, it can be found that the Accuracy, macro-F1, and AUC of the VWI-DNN algorithm do not change much with the increase of the pace, which is relatively stable.

(a)

(b)

(c)

3.3. Discussion

This study demonstrated the capability of the proposed system to detect gait phases based on acceleration signals. To support this hypothesis, this paper proposes to use the voting-weighted integrated neural network to identify the gait phase and compares it with other integrated learning algorithms to verify the effectiveness of the algorithm.

3.3.1. Acceleration Signals Analysis

Walking activity emanated from the human body is important, and this information can be extracted through the use of acceleration signals. Although the VWF-DNN algorithm has shown certain validity in the classification of acceleration signals for gait event detection, it still needs to be further optimized in the future. In this study, the inertial sensor module needed to be placed at a designated location on the instep, lower leg, and thigh of each subject. However, due to the height, weight, gender, etc. of each subject, the sensors cannot be accurately placed in the specified position and can only be installed at an approximate designated position, which needs further investigation [43].

To characterize acceleration signals, there are three main cascaded modules which are data processing, feature extraction, and classification methods. It should be noted that the classification accuracy depends greatly on the features extracted. Moreover, four combined features is better than a single or two features for upper limb movement. This shows the network responses of five TD features. By observing Figures 8–10, it can be seen that all the training functions have clearly classified the HS and SW phases and there is very low classification error between the two. Interestingly, we can see through the confusion matrix that the Bagging and AdaBoosting misinterpreted the FF phase and HO phase, which results in a low-recognition precision for the FF and HO phases. Also, by analyzing the results of Tables 3–5, it can be shown that the model has a poor recognition effect on the FF and HO phases. This may be due to the fact that the two adjacent phase features are too similar. In addition, Figures 8–10 also illustrate that the phase difference between the FF and HO phase characteristics are not obvious, but the proposed VWI-DNN algorithm in this paper can significantly improve this situation.

Regarding the effective classification of acceleration signals, the five TD features provide better classification of gait events and the average accuracy of the VWF-DNN algorithm for HS, FF, HO, and SW phases is 99.2%. This study was then compared to some of the previous research studies. With regard to the gait event, different walking conditions by IMU located on foot show 82.2% accuracy using the ANN [39]. Similar result is achieved when five gait phases were classified using IMUs 82% accuracy [40]. In addition, this is important for the development of assistive devices for the lower leg, as they have strong relationships with the gait event [41]. To propose a system that may be applied to any individual, the generalization of trained VWF-DNN algorithm was tested on unlearned data of acceleration signals. This study found that the proposed system could predict the gait event successfully for the unlearned data. In general, the detection of HS, FF, HO, and SW phases based on acceleration signals seemed reliable.

In reviewing the literature, the reliability of IMUs was questioned. It is interesting to note that the percentage of stance and swing phases of footswitch data was in line with the theory of the gait cycle as 60% of the complete cycle was the stance phase while the remaining percent was swing phase 40% [42]. According to Observation Tables 3 and 4, it can be found that the recognition of FF and HO phases by Bagging and AdaBoosting is generally low, while other two phases is very high. In addition, according to observations of Figures 8–10, we can also see that Bagging and AdaBoosting divided FF phase into HO phase and also divide HO phases into FF phases. These results indicate that the features we extracted may require further discussion.

3.3.2. Gait Phase Detection

The purpose of this study was to apply a machine learning to predict HS, FF, HO, and SW from acceleration signals. In support of the hypothesis aforementioned, this study proposes the VWI-DNN algorithm and uses it to successfully predict HS, FF, HO, and SW phases. The results of learned data indicated that the acceleration signals had low variability and stability during walking on the ground. Compared with the research carried out by Nazmi et al. [1], this study can obtain higher recognition accuracy, but the neural network model used in this study was too complicated, which may lead to longer training time. Some portable gait event detection applications require functional electronic simulators, dynamic gait monitoring, and gait biofeedback [44], but currently no wearable sensors meet these requirements.

The VWI-DNN algorithm used a voting mechanism to fuse results and used three subneural networks to vote. Whether the three subneural networks are set up properly needs further exploration. The selection of the bonus penalty function takes the derivative changes and actual effects into account, but it still avoids the occasional occurrence of improper rewards and punishments, resulting in the final classification results are not ideal. Nevertheless, the model used in this paper can still achieve the recognition accuracy and the macro-F1 value, which were higher than 98%. And as can be seen from Figure 13, the AUC value of VWI-DNN algorithm reaches the maximum value of 1. It can be seen from the performance of each classifier in the AUC that the classification result obtained by the VWI-DNN algorithm is more reliable.

In terms of the generalization of the proposed system, this study revealed the VWI-DNN algorithm achieved better performance in gait phase recognition. The VWI-DNN algorithm based on the voting weighting mechanism detects HS, FF, HO, and SW phases with higher recognition accuracy, macro-F1, and AUC than existing Bagging and AdaBoosting. However, the macro-F1 of the FF and HO phases obtained by Bagging and AdaBoosting are very low. According to Figures 8–10, these results indicate that the HS and SW phase was reasonably accurate and FF and HO phases should yield more warnings. Even so, using the VWI-DNN algorithm proposed in this paper still obtains a good recognition effect on the FF and HO phases, and in some sense compensates for the defects of dividing the two phases. The study also shows that the proposed use of vector difference to distinguish between ST and IL phases does have a certain effect. In addition, it is possible to explain that the VWI-DNN algorithm proposed in this paper has strong performance in the field of gait phase division.

3.3.3. Limitations

There are some limitations to this study. Even though VWI-DNN has shown its usefulness in classifying acceleration signals for gait event detection, a further evaluation is needed using other machine learning approaches. Future works should improve the classification accuracy by improving the method of extracting features, gait phase recognition algorithm, etc. In this study, the wearing three inertial sensor modules were assumed to be an acceptable wearable sensor compared to other wearable sensors. However, the wearing of the sensor in practice may have a potential impact on the gait of the person who has not yet been investigated.

The detection of gait phase in this study relied on data collected by IMUs. Although acceleration sensors have low cost and fast dynamic response, the accuracy might be affected by the circuit design and placement of sensors. Different walking styles could lead to erroneous placement of IMUs. However, as the sensor processing advances and algorithm innovations, these errors are further reduced. Therefore, IMU can possibly be applied in large scale in the direction of gait detection in the near future.

4. Conclusion

In order to recognize the walking gait phase accurately, this work proposed a VWI-DNN model to analyze multidimensional acceleration signals and detect different gait events including HS, FF, HO, and SW. It consists of three main parts, data preprocessing, multistream integrated neural network, and voting-weighted function, where data preprocessing employs PCA dimensionality reduction, four-phase division, and key feature selection in time domain. In addition, multiple refined DNNs are applied to design a multistream integrated neural network, which utilizes the mixture-granularity information to form a high-dimensional feature. Finally, a voting-weighted function is developed to fuse dissimilar submodels as a unified representation for distinguishing small discrepancy among different gait phases. It constructs a general neural network allowing for fusing different voting decisions at each submodel and demonstrates its use as a practical way to massively increase model capacity. Experiments and discussion demonstrate the effectiveness of the VWI-DNN with higher accuracy and macro-F1 up to 99.5%, which outperform other voting methods.

The network generates large number of parameters, which increases the time for model classification. Therefore, the model can only be trained on the GPU and the IMU data cannot be classified online on a convenient mobile device. Our future work is to try a lightweight network to compress the model parameters and speed. And we will attempt to combine our method with the newly updated work for other gait phase detection applications, such as rehabilitation training robot and medical Internet of Things.

Data Availability

The acceleration data used to support the findings of this study have not been made available since these data are related to the personal privacy of each volunteer involved in the experiment.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This study was financially supported by the Fundamental Research Funds for the Central Universities (No. 2015ZCQ-GX-03), National Key Research and Development Program of China (No. 2017YFC1600605), National Natural Science Foundation of China (No. 61673002), and Beijing Municipal Education Commission (No. KM201910011010).

References

N. Nazmi, M. A. Abdul Rahman, S.-I. Yamamoto, and S. A. Ahmad, “Walking gait event detection based on electromyography signals using artificial neural network,” Biomedical Signal Processing and Control, vol. 47, pp. 334–343, 2019.
View at: Publisher Site | Google Scholar
Y. Okubo, D. Schoene, and S. R. Lord, “Step training improves reaction time, gait and balance and reduces falls in older people: a systematic review and meta-analysis,” British Journal of Sports Medicine, vol. 51, no. 102, pp. 586–593, 2017.
View at: Publisher Site | Google Scholar
S. Tian, M. Li, Y. Wang, and X. Chen, “Application of an improved correlation method in electrostatic gait recognition of hemiparetic patients,” Sensors, vol. 19, no. 11, p. 2529, 2019.
View at: Publisher Site | Google Scholar
T. Yan, M. Cempini, C. M. Oddo, and N. Vitiello, “Review of assistive strategies in powered lower‐limb orthoses and exoskeletons,” Robotics and Autonomous Systems, vol. 64, pp. 120–136, 2015.
View at: Publisher Site | Google Scholar
H. Vu, F. Gomez, P. Cherelle, D. Lefeber, A. Nowé, and B. Vanderborght, “ED-FNN: A new deep learning algorithm to detect percentage of the gait cycle for powered prostheses,” Sensors, vol. 18, no. 7, p. 2389, 2018.
View at: Publisher Site | Google Scholar
S. Murray and M. Goldfarb, “Towards the use of a lower limb exoskeleton for locomotion assistance in individuals with neuromuscular locomotor deficits,” in Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 1912–1915, San Diego, USA, August 2012.
View at: Publisher Site | Google Scholar
S. Ounpuu, “The biomechanics of walking and running,” Clinics in Sports Medicine, vol. 13, pp. 4843–4863, 1994.
View at: Google Scholar
A. Ferrari, R. Brunner, S. Faccioli et al., “Gait analysis contribution to problems identification and surgical planning in CP patients: an agreement study,” European Journal of Physical & Rehabilitation Medicine, vol. 51, no. 1, pp. 39–48, 2015.
View at: Google Scholar
J. Taborri, E. Scalona, S. Rossi et al., “Real-time gait detection based on hidden Markov model: is it possible to avoid training procedure?” in Proceedings of the 2015 IEEE International Symposium on Medical Measurements and Applications, pp. 141–145, Torino, Italy, May 2015.
View at: Google Scholar
J. Mukherjee, P. Chattopadhyay, and S. Sural, “Information fusion from multiple cameras for gait-based re-identification and recognition,” IET Image Processing, vol. 9, no. 11, pp. 969–976, 2015.
View at: Google Scholar
S. Rosati, V. Agostini, M. Knaflitz, and G. Balestra, “Muscle activation patterns during gait: a hierarchical clustering analysis,” Biomedical Signal Processing and Control, vol. 31, pp. 1746–8094, 2017.
View at: Publisher Site | Google Scholar
S. Ding, X. Ouyang, Z. Li, and H. Yang, “Proportion-based fuzzy gait phase detection using the smart insole,” Sensors and Actuators A: Physical, vol. 284, 2018.
View at: Publisher Site | Google Scholar
Y. Shimada, S. Ando, T. Matsunaga et al., “Clinical application of acceleration sensor to detect the swing phase of stroke gait in functional electrical stimulation,” The Tohoku Journal of Experimental Medicine, vol. 207, no. 3, pp. 197–202, 2005.
View at: Publisher Site | Google Scholar
N. Chia Bejarano, E. Ambrosini, A. Pedrocchi, G. Ferrigno, M. Monticone, and S. Ferrante, “A novel adaptive, real-time algorithm to detect gait events from wearable sensors,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 23, no. 3, pp. 413–422, 2015.
View at: Publisher Site | Google Scholar
R. Caldas, M. Mundt, W. Potthast, F. Buarque de Lima Neto, and B. Markert, “A systematic review of gait analysis methods based on inertial sensors and adaptive algorithms,” Gait & Posture, vol. 57, pp. 204–210, 2017.
View at: Publisher Site | Google Scholar
M. D. Sánchez Manchola, M. J. Pinto Bernal, M. Munera, and C. A. Cifuentes, “Gait phase detection for lower-limb exoskeletons using foot motion data from a single inertial measurement unit in hemiparetic individuals,” Sensors, vol. 19, no. 13, 2019.
View at: Publisher Site | Google Scholar
M. Yuwono, S. W. Su, Y. Guo, B. D. Moulton, and H. T. Nguyen, “Unsupervised nonparametric method for gait analysis using a waist-worn inertial sensor,” Applied Soft Computing, vol. 14, pp. 72–80, 2014.
View at: Publisher Site | Google Scholar
Y. Bai, X. Jin, X. Wang, T. Su, J. Kong, and Y. Lu, “Compound autoregressive network for prediction of multivariate time series,” Complexity, vol. 2019, Article ID 9107167, 11 pages, 2019.
View at: Publisher Site | Google Scholar
J. Taborri, S. Rossi, E. Palermo, F. Patanè, and P. Cappa, “A novel HMM distributed classifier for the detection of gait phases by means of a wearable inertial sensor network,” Sensors, vol. 14, no. 9, pp. 16212–16234, 2014.
View at: Publisher Site | Google Scholar
R. Liu, J. Zhang, L. Wang, and M. Zhang, “Research on face recognition method based on combination of SVM and LDA-PCA,” in The Proceedings of the Second International Conference on Communications, Signal Processing, and Systems, pp. 62–66, Springer, Berlin, Germany, 2014.
View at: Google Scholar
X. Lu and X. Xu, “Human behavior recognition based on acceleration and hga-bp neural network,” Computer Engineering, vol. 41, no. 9, pp. 220–232, 2015.
View at: Google Scholar
J. Ma and Y. Yuan, “Dimension reduction of image deep feature using PCA,” Journal of Visual Communication and Image Representation, vol. 63, 2019.
View at: Publisher Site | Google Scholar
T. Juri, P. Eduardo, R. Stefano et al., “Gait partitioning methods: a systematic review,” Sensors, vol. 16, no. 1, pp. 66–91, 2016.
View at: Google Scholar
J. Kim, S. Hwang, R. Sohn, Y. Lee, and Y. Kim, “Development of an active ankle foot orthosis to prevent foot drop and toe drag in hemiplegic patients: a preliminary study,” Applied Bionics and Biomechanics, vol. 8, no. 3-4, pp. 377–384, 2011.
View at: Publisher Site | Google Scholar
N. B. Bolus, C. N. Teague, O. T. Inan, and G. F. Kogler, “Instrumented ankle-foot orthosis: towards a clinical assessment tool for patient-specific optimization of orthotic ankle stiffness,” IEEE/ASME Transactions on Mechatronics, vol. 22, no. 6, pp. 2492–2501, 2017.
View at: Publisher Site | Google Scholar
C. Mummolo, L. Mangialardi, and J. H. Kim, “Quantifying dynamic characteristics of human walking for comprehensive gait cycle,” Journal of Biomechanical Engineering, vol. 135, p. 91006, 2013.
View at: Publisher Site | Google Scholar
X. Jin, N. Yang, X. Wang, Y. Bai, T. Su, and J. Kong, “Integrated predictor based on decomposition mechanism for PM2.5 long-term prediction,” Applied Sciences, vol. 9, no. 21, p. 4533, 2019.
View at: Publisher Site | Google Scholar
Y.-Y. Zheng, J.-L. Kong, X.-B. Jin, X.-Y. Wang, T.-L. Su, and J.-L. Wang, “Probability fusion decision framework of multiple deep neural networks for fine-grained visual classification,” IEEE Access, vol. 7, pp. 122740–122757, 2019.
View at: Publisher Site | Google Scholar
Y. Y. Zheng, J. L. Kong, X. B. Jin, X.-Y. Wang, and M. Zuo, “Cropdeep: the crop vision dataset for deep-learning-based classification and detection in precision agriculture,” Sensors, vol. 19, no. 5, p. 1058, 2019.
View at: Publisher Site | Google Scholar
A. N. M. Jaya Lakshmi and K. V. Krishna Kishore, “Performance evaluation of DNN with other machine learning techniques in a cluster using Apache Spark and MLlib,” Journal of King Saud University Computer and Information Sciences, 2018.
View at: Publisher Site | Google Scholar
J. Liu, S. Xia, and W. Yu, “A neural network integrated incremental learning method for voting right value adjustment,” Signal Processing, vol. 26, no. 1, pp. 46–50, 2010.
View at: Google Scholar
N. Tommaso, C. Renzo, and A. Claudio, “An integrated artificial neural network–unscented Kalman filter vehicle sideslip angle estimation based on inertial measurement unit measurements,” Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, vol. 233, no. 7, pp. 1864–1878, 2019.
View at: Publisher Site | Google Scholar
D. Povey, X. Zhang, and S. Khudanpur, “Parallel training of deep neural networks with natural gradient and parameter averaging,” 2014, https://arxiv.org/abs/1410.7455.
View at: Google Scholar
G. Zhou, S. Ge, and L. Yang, “Fault diagnosis method of nuclear power plant based on neural network and vote fusion,” Journal of Atomic Energy Science and Technology, vol. 44, no. B09, pp. 367–372, 2010.
View at: Google Scholar
J. Zhu, A. Arbor, and T. Hastie, “Multi-class AdaBoost,” Statistics and Its Interface, vol. 2, no. 3, pp. 349–360, 2006.
View at: Google Scholar
L. Breiman, “Bagging predictors,” Machine Learning, vol. 24, no. 2, pp. 123–140, 1996.
View at: Publisher Site | Google Scholar
W. W. Rodrigo and H. S. Lopes, “Neural networks for protein classification,” Applied Bioinformatics, vol. 3, no. 1, pp. 41–48, 2004.
View at: Google Scholar
R. H. W. Pinheiro, G. D. C. Cavalcanti, R. F. Correa, and T. I. Ren, “A global-ranking local feature selection method for text categorization,” Expert Systems with Applications, vol. 39, no. 17, pp. 12851–12857, 2012.
View at: Publisher Site | Google Scholar
H.-Y. Lau, K.-Y. Tong, and H. Zhu, “Support vector machine for classification of walking conditions of persons after stroke with dropped foot,” Human Movement Science, vol. 28, no. 4, pp. 504–514, 2009.
View at: Publisher Site | Google Scholar
D. Sanchez-Valdes, A. Alvarez-Alvarez, and G. Trivino, “Walking pattern classification using a granular linguistic analysis,” Applied Soft Computing, vol. 33, pp. 100–113, 2015.
View at: Publisher Site | Google Scholar
J. L. Bartlett and R. Kram, “Changing the demand on specific muscle groups affects the walk-run transition speed,” Journal of Experimental Biology, vol. 211, no. 8, pp. 1281–1288, 2008.
View at: Publisher Site | Google Scholar
J. Taborri, E. Palermo, S. Rossi, and P. Cappa, “Gait partitioning methods: a systematic review,” Sensors, vol. 66, pp. 1–20, 2016.
View at: Google Scholar
F. Provost, T. Fawcett, and R. Kohavi, “The case against accuracy estimation for comparing induction algorithms,” in Proceedings of the Fifteenth International Conference (ICML 98), J. Shavlik, Ed., pp. 445–453, San Francisco, CA, USA, 1998.
View at: Google Scholar
M. Hanlon and R. Anderson, “Real-time gait event detection using wearable sensors,” Gait & Posture, vol. 30, no. 4, pp. 523–527, 2009.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2020 Lei Yan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

3292

Downloads

1872

Citations