Abstract
We developed a knee brace to measure the knee angle and implicitly the flexion/extension (f/e) of the knee joint during daily activities. The goal of this study is to classify and validate a limited set of physical activities on ten young healthy subjects based on knee f/e. Physical activities included in this study are walking, ascending and descending of stairs, and fast locomotion (such as jogging, running, and sprinting) at self-selected speeds. The knee brace includes 2 accelerometers for static measurements and calibration and an inductive sensor for dynamic measurements. As we focus on physical activities, the inductive sensor will provide the required information on knee f/e. In this study, the subjects traversed a predefined track which consisted of indoor paths, outdoor paths, and obstacles. The activity classification algorithm based on peak detection in the knee f/e angle resulted in a detection rate of 95.9% for walking, 90.3% for ascending stairs, 78.3% for descending stairs, and 82.2% for fast locomotion. We conclude that we developed a measurement device which allows long-term and ambulatory monitoring. Furthermore, it is possible to predict the aforementioned activities with an acceptable performance.
1. Introduction
Physical activity is healthy as it prevents or reduces chronic health issues like obesity, diabetes, heart diseases, and depression. Therefore, physical activity as well as encouraging physical activity is gaining importance in the therapy and prevention of these diseases.
The last decades witnessed a lot of engineering and research efforts is put into developing small, cheap, and accurate wearable sensing devices. Today, these wearable sensors have reached the point that they can be used for clinical application [1, 2]. In gait analysis wearable sensors are used to measure motion and they are worn at different parts of the subject’s body. Commonly used sensors are accelerometers, gyroscopes, force sensors, strain gauges, inclinometers, and electrogoniometers [3]. The signals measured by these wearable sensors can be used to monitor movement and perform gait analysis. However, before actual gait analysis can be performed the raw sensor signals must be processed. Mathematical and statistical methods such as curve sketching [4, 5], fuzzy logic algorithms [6–8], neural networks [9–11], and support vector machines [12, 13] are often used to extract features and classify them into certain gait phases or activities [3].
Next to the sensors itself, a lot of wearable devices for measuring physical activity are already available, such as ActiGraph (Pensacola, USA), GeneActive (Activinsights, UK), Dynaport (McRoberts, The Netherlands), CAM (Maastricht Instruments, The Netherlands), and ActivePal (PAL Technologies, UK). These devices exist of one or more of the previously described sensors in combination with an algorithm which translates the sensor data into activity information. They are effective in monitoring human activity [14–19] as they are small, inexpensive, and easy to wear. Nevertheless, these devices are not capable of determining the way the physical activity was performed by, for example, extracting the knee flexion/extension (f/e) angles during outdoor walking.
Sophisticated and well-equipped gait laboratories are often involved when objective information on the performance of physical activities is required. These gait laboratories can accurately measure gait parameters such as joint angles during training, medical diagnosis, or rehabilitation. However, these gait laboratories are expensive and require qualified and trained personnel to operate. Furthermore, these measurements are restricted to the laboratory environment. This implies that the subject performs instructed activities raising the question whether the results can be generalized to daily (real-life) activities [20].
To overcome the restrictions and disadvantages related to gait laboratories, several attempts are described in the literature to use accurate and wearable kinematic monitoring systems. Tesconi et al. [21] presented a wearable kinesthetic system for knee f/e monitoring by conductive elastomeric composites (CEs) integrated in a fabric band which is wrapped around the knee joint. This system discriminates healthy subjects from injured subjects. Kobashi et al. [17] reported a wearable joint kinematic monitoring system including accelerometers and a rate gyroscope with miniature size, low power consumption, and wireless data transmission. A similar system is proposed by Huddleston et al. [22] to test the accuracy of a modified Intelligent Device for Energy Expenditure and Activity (IDEEA) in the measurement of knee f/e angles. The advantage of these two systems is that they can measure f/e angles with an accuracy comparable to a laboratory setup, but a drawback is that they require several devices worn by the subject. This is not suitable for long-term and ambulatory monitoring as these devices have to be placed on an exact location on the body. Furthermore, these devices have to be connected with each other. This implies the usage of a lot of wires which is often experienced as uncomfortable and disturbing by the subject. Finally, Yalcin et al. [23] evaluated several methods for recognizing walking activities using on-body wireless nodes equipped with inertial and orientation sensors. Their results show that the magnetometer sensor performs best, regardless of the classification method and sensor position. Both the magnetometers and gyroscopes significantly outperform the accelerometer.
The currently available wearable systems with a high accuracy mainly exist as several and complex wired electronic devices that are bound to a correct placement on the body. Furthermore, when using accelerometers and gyroscopes one should know that initial joint angles and an accumulation of the integral error are significant problems especially when applied for long-term monitoring. To be able to monitor activities for a long term and in real-life circumstances, we developed a new device. This device is integrated in a knee brace and consists of a self-inductance sensor and two accelerometers to measure the f/e of the knee joint [24, 25].
This inductive sensor technology has advantages compared to current activity monitors. The sensor is a coil which can be incorporated in, or worn under, clothing. Furthermore, the output signal is one dimensional and directly related to the measured joint angle. For this reason, it is straightforward to process compared to other complex signals as, for instance, produced by triaxial accelerometers. These triaxial accelerometers perform poorer under dynamic conditions as the gravitational component is superimposed on the component of the actual movement. Next, the self-inductance sensor is also not prone to drift like, for example, gyroscopes are. Finally, no complex filtering or fusing algorithms are required to process the self-inductance sensor data. This will eventually improve the feasibility for long-term and ambulatory monitoring and can be a first step in measuring how activities are performed.
Based on this self-inductance sensor we developed a classification algorithm which can detect strides during normal walking, fast locomotion (such as jogging, running, and sprinting) and descending and ascending stairs in real-life and ambulatory circumstances.
In this paper, we will first present the knee brace measuring system which is based on the inductive bending sensor technology. Next, we will describe how activities are classified based on the output of the inductive sensor. Finally, we will present a validation study on ten healthy subjects with normal gait, outside the laboratory environment.
2. Materials and Methods
2.1. Knee Brace with Inductive Sensor Technology
In order to measure activities in real-life circumstances we need a sensor which is easy to wear and which can measure the angle of the knee joint. Moreover, this sensor should fit easily, and no strict placement should be required as the subject needs to put it on without assistance. Moreover, to enhance the feasibility for real-life and long-term monitoring the output signal may not be complex and may not be prone to drift or noise. Therefore, we developed a knee brace (KB) to measure f/e of the knee joint in an ambulatory setting during daily life, rehabilitation, or training. This KB is easy to wear and can measure knee f/e. The KB, designed by TNO Medical Devices [24, 25], includes two accelerometers and a coil (Figure 1).
Measuring the f/e angle of the knee joint, or more generally using measuring bending of the knee brace and the coil attached to it, is based on the change of the electrical inductance of a coil (a loop of a multicore conductive wire with a diameter of 400 μm). It means that the inductance of a coil changes according to a change in its form (Figure 2). The mutual inductance by a filamentary circuit on a filamentary circuit is given by the double integral where denotes the magnetic constant ( H/m), and are curves along the wires, and is the distance between two points on and , respectively. The vectors and represent vectors along and . When equals zero, the previous equation becomes infinite. Therefore, an extra constraint is introduced that states that has to be larger than half the thickness of the wire . In that case the inductance is only dependant on the radius , its length , and the factor denoting the current distribution through the wire (normally ). When the form of the coil changes, the orientation of and and their inter vector distance will change accordingly. This results in a change of the inductance of the coil. To measure this inductance, the coil will be combined with a capacitor to create an LC-circuit which is placed in oscillation mode. This LC-circuit, fed by an AC-current (<2 mA), generates an oscillation frequency depending on the coil’s inductance value. The coil has an inductance ranging from 0 nH to 500 nH, resulting in an according oscillation frequency between 2.4 MHz and 1.9 MHz. Notice that a fixed inductance has been used to create an offset in the frequency reading. The oscillation frequency is measured directly by a microcontroller and is sent to the data logger and registered in a file. This means that we register the inductance implicitly by measuring the frequency. In a later stage we can translate this frequency into inductance.
As it can be seen in (1), the mutual position and orientation of the wire segments determine the total inductance of the coil. The position and orientation of segments of the wire change according to the bending of the wire and consequently change the inductance of the coil. A more intuitive approach is provided in Figure 2.
The wire used in this sensor is a very flexible and thin wire integrated in a carrier, for example, a knee brace. In case the bend of the coil is directly connected to a single bending angle, that is, of a single human joint (e.g., knee or elbow), a simple calibration can be used to translate the inductance readings into an absolute angular value. The calibration can be done using a reference measuring system. However, in some cases the carrier may be minimally shifted due to movement. The system can then be recalibrated based on extra information from auxiliary accelerometers placed on the knee brace (Figure 1). These two triaxial accelerometers are placed at the ligaments of the joint. When the subject is in a steady state (sitting, standing, etc.) the readings of the accelerometers may be used to calculate the absolute angle of the joint. In static situation the accelerometers are only measuring the gravitation force. Based on that, the angle between the accelerometer is calculated. In dynamic situations (walking, running, etc.), the accelerometers “see” simultaneously the gravitation and inertial forces of the human moving. In that case an accurate calculation of the joint angle is not possible using the accelerometers. Then the inductance measurement is used for calculating the joint angle. This calibration technique is patented in “Measuring the angle between a first and a second member under dynamic conditions” (WO 2009061181 (A1), EP 2057944 (A1), US 2010286950 (A1), EP 2219521 (A1), EP 2219521 (B1)).
To validate this new measurement setup Gransier et al. [26] and Meijer et al. [27] conducted an experiment where this sensor technology and calibration technique is compared to a laboratory golden reference system, the optical motion capture system (Vicon, Vicon Motion Systems, UK). They [26, 27], measured a Root Mean Square Error of 3.77° (Standard Deviation (SD) 1.44°) and a Mean Absolute Error of 2.77° (SD 1.02°) regarding knee angles in the sagittal plane. Gransier et al. [26] concluded that the measured accuracy of this sensor technology and calibration technique compared to a laboratory golden reference system is within clinically acceptable standards, considering a clinical cut-off point of 5 degrees of deviation or more [28].
As the bend of the coil is directly connected to a single bending angle of the knee joint and as we focus on physical activities (dynamic conditions), the inductive sensor provides the required information on knee f/e. We can conclude that we can measure knee f/e using the coil technology as an electrogoniometer. In this setting (by omitting the accelerometers) we obtain relative joint angles. If absolute angles are required, two accelerometers should be added in order to calibrate the coil. From this point on we only consider the coil technology and the relative f/e angles.
2.2. Study Protocol
This study included ten young and healthy subjects (6 males and 4 females, mean age 27.7 yrs, SD 3.1 yrs). Subjects with orthopedic aids were excluded (except for arch support) during the measurement. Each subject included in this study has read and signed an informed consent conforming to the guidelines of the commission of medical ethics of KU Leuven (Belgium).
First, each subject was asked to put on the knee brace by himself to the left knee joint. The left joint is preferred as in this setup the data logger is placed at the left side of the body where it is less disturbing during the activities. Next, a predefined track was traversed twice. Firstly, a practice trial was performed to get used to the knee brace and to experience the track. Secondly, the actual measurements were performed. The track consisted of indoor (60%) and outdoor (40%) paths including obstacles such as stairs (10%), slopes (2%), and different surfaces like paved road (6%), grass, and gravel (8%). At the end of the track the subjects were asked to increase their locomotion in three different levels, namely, jogging, running and sprinting. Additionally, each track traverse was filmed to be able to label the activities.
The mean and SD of the number of activities/strides performed in one trial are given in Table 1. These are the number of activities labeled based on the video images. This shows that subjects traversing the same track can perform a different number of strides. For instance, the number of stair ascends/descents depends on which leg is used to take the first stair. Note that the activity with fast locomotion includes jogging (slow running), running, and sprinting (fast running) at self-selected speeds.
2.3. Classification Algorithm
We developed a classification algorithm which can detect strides during normal walking, fast locomotion (such as jogging, running, and sprinting), stairs descend, and stairs ascend in real-life and ambulatory circumstances. This activity classification algorithm is based on peak detection in the knee f/e angle signal measured by the inductive coil of the knee brace. We decided to only make use of the raw signal generated by the inductive sensor as this is a relative measure directly related to the joint angle. The classification algorithm will not use the calibrated signal (which is expressed in absolute joint angles) as we therefore need both accelerometers to calibrate the inductive sensor signal. We use the uncalibrated data because we want to keep the number of sensors required to classify the activity as low as possible. One sensor is easier to incorporate in clothing and the sensor output is easier to process in comparison with using all three sensors. Nevertheless, the knee brace records signals from both accelerometers and the coil simultaneously which allows us to plot absolute joint angles, which makes it easier to interpret. When referring in this paper to the f/e angle we mention angle α as shown in Figure 3(b).
(a)
(b)
Before the actual classification can be performed, some preprocessing steps are required. First, a mean filter over 10 samples is applied, which results in a smoothed signal containing local minima and maxima as shown in Figure 3(a) for one stride.
From these local minima and maxima we can extract interesting features. We can see that one stride can be determined by two minima below a certain threshold. This threshold cannot be fixed to a certain angle as this is person specific. Therefore we continue with determining the period of each individual stride/activity by indicating the local minima which bound this period. The other, minima will be removed because in the further processing steps these minima are not relevant.
A minimum (Figure 4) has to be removed if the following holds: When is evaluated over all minima in Figure 4, this will result in being removed. and are kept as they indicate the boundary of one activity period.
After these preprocessing steps each individual activity is bounded by two successive minima. The classification algorithm, based on a decision tree (Figure 5), has now to distinguish between the different activities.
First in the decision tree, we evaluate the time between two successive local minima. In other words, we calculate the stride duration and therefore also the number of steps per minute or cadence.
Tudor-Locke et al. [29] reported about cadence during daily-life activities based on the results of the US National Health and Nutrition Exam Survey (NHANES). The NHANES took place over the years 2005-2006 where 3,744 adults (20 years or older) were measured using an ActiGraph (Pensacola, USA) which outputs cadence as a function of time. Tudor-Locke et al. determined different cadence bands, which are 1–19 steps/minute (incidental movement), 20–39 steps/minute (sporadic movement), 40–59 steps/minute (purposeful steps), 60–79 steps/minute (slow walking), 80–99 steps/minute (medium walking), 100–119 steps/minute (brisk walking), and 120+ steps/minute (encompassing all faster human locomotion movements). Based on these results of Tudor-Locke, a first threshold can be set in the decision tree. As we want to detect normal walking, or medium walking referring the cadence bands reported by Tudor-Locke et al., this means that activities with a cadence less than 80 steps/min, hence, a period larger than 1.5 s, will be classified as “no detection.” Next, to distinguish between normal/medium walking and fast locomotion (like jogging, running, and sprinting), the threshold of 120 steps/minute proposed by Tudor-Locke et al. is experienced as too low. The subjects participating in this study already reached a cadence of 120 steps/minute during normal or medium walking. By the authors knowledge there is currently no literature available where cadence bands are defined for jogging, running, and sprinting in healthy subjects. In general, we assume that fast locomotion, like jogging, running, and sprinting, for healthy subjects is performed at +150 steps/minute. Egerton et al. [30] reported about the relationship between stride length and cadence. They concluded that at very high cadence (+150 steps/min) stride lengths begin to reduce. This cut-off indicates that an activity performed with a cadence of 150 steps/min or higher can be classified as fast locomotion. Therefore, activities with a cadence of +150 steps/min, hence, a period shorter than 0.8s, will be classified as “fast locomotion.” Activities within the range of 80–150 steps/minute will be classified as normal walking.
Next to the cadence, we further distinguish between the activities by counting the number of maxima within one period. More than two maxima in one period are unknown activities and therefore classified as “no detection.” Exact two maxima indicate that a step or running was performed as illustrated in Figure 7(a) and Figure 7(d). One maximum is typical for stair ascend or stair descent. If the maximum is reached in the first half of the period the subject goes downstairs. On the other hand, if the maximum is reached in the last half of the period, the subject goes upstairs. Notice that we do not include running downstairs or upstairs as this was out of the scope of this study.
Figure 7 shows the f/e angle pattern of the four different activities indicating their local minima and maxima.
3. Results and Discussion
The proposed system and classification algorithm was tested in ten young healthy subjects. The knee brace with the integrated inductive sensor technology was able to record the knee joint f/e in an ambulatory setting. Each measurement was labeled by means of video recordings and the algorithm was able to distinguish the four assumed activities.
Table 2 reports on the confusion matrix that summarizes the results, in terms of percentage of labeled activities predicted by the algorithm belonging to a certain activity. In the first row we can see that 95.9% of the video labeled as normal walking (strides) was indeed detected (predicted) as normal walking (strides) by the algorithm. Furthermore, on the first row, one can see that 0.9% of the activities labeled as normal walking (strides) are classified by the algorithm as stair ascend.
High values on the diagonal of the confusion matrix indicate that the algorithm can make a good distinction between the different activities.
Furthermore, we observed that different terrains or slopes do not have a significant effect on the classification results. Figure 6 shows the f/e pattern for indoor and outdoor walking steps for one subject extracted from one arbitrary chosen dataset. The outdoor steps shown in Figure 6 were preformed when walking on a rough lawn. When looking at the minima (the ones which determine the periods of each stride) we observed a significant difference between the indoor and outdoor walking pattern. For indoor walking the mean of the minima (knee flexion angle) is 131.9 (SD 0.76°). On the other hand, for outdoor walking the mean of the minima (knee flexion angle) is 117.2° (SD 2.78°). Nevertheless, with this significant difference in walking patterns the classification algorithm will classify all these steps as “normal walking.” This can be explained by the similar shape of both steps. As the algorithm does not take absolute angles of the knee f/e but only looks at the relative values (or in other words it looks at the shape of the signal) these steps will be classified correctly.
(a)
(b)
(c)
(d)
Nevertheless, when walking on a lawn or another rough and uneven surface, one can make a movement which looks like, and therefore also is predicted as, for example, stair ascend but which is video labeled as walk. This is only a small portion of the wrongly predicted activities and they occur as isolated events.
In two of the ten trials, the knee brace came down. This resulted in an amplitude change of the inductive sensor signal. But as this algorithm is independent of the absolute value, the activities were still correctly detected. To quantify these observations we conducted a second experiment. In this experiment three young healthy subjects traversed a shortened indoor path including normal walking, stair ascend and descent, and fast locomotion. Each subject traversed the path three times. The first time the KB was put on as it normally should be. Next, the KB was put on in the lowest possible position. In the third and final run, the KB was put on in the highest possible position. This resulted in a confusion matrix provided in Table 3. The position of the KB in the 2nd and 3rd run is provided in Table 4.
The group of “no detection” in Table 2 which means that there was an activity labeled on the video but that the algorithm does not predict any activity includes the largest part of the misclassified activities. By inspecting the video labeling the “no detections” can be seen as transitions like, for example, taking the first or last stair and approaching a door or doorstep. From Table 3 we can observe that fast locomotion is often misclassified as normal working.
4. Conclusion
In this study we wanted to measure activities in real-life circumstances and automatically classify them into four categories: normal walking, stair ascend, stair decent, and fast locomotion. Therefore, we developed a new sensor. This sensor incorporates a knee brace extended with a self-inductance sensor to measure flexion/extension (f/e) of the knee joint during daily activities, rehabilitation, or training in an ambulatory setting.
As shown in the confusion matrix (Table 2), a good distinction can be made between the aforementioned activities. An accuracy is reached of 95.9% for normal walking, 90.3% for stair ascend, 78.3% for stair decent, and 82.2% for fast locomotion. We can conclude that it is possible to predict the performed activities in an ambulatory setting with an acceptable performance. The position of the brace slightly influences the accuracy of the classification but still an acceptable performance is reached.
Nevertheless, the performance accuracy of the knee brace can be improved by reducing some of the misclassifications. Most of the misclassifications can be found in the group of “no detection.” These are activities labeled on the video, which the algorithm does not detect at the same time in the knee f/e signal. These activities are most of the time transitions like, for example, a subject approaching a door or doorstep, someone crossing the path of the subject and the subject having to change his gait or the subject taking the first stair. These result in a knee f/e pattern which does not fit the proposed activity patterns (Figure 7) and it will therefore be classified as “no detection.” To reduce these misclassifications a strict description of each activity is required, including the cut-off where a normal activity changes in a transitional activity.
Furthermore, there are also some false detections at stair decent. This can be explained by the fact that subjects who are familiar with the stairs are going downstairs faster and in a more normal walking or frolicking way. This results in f/e angles with a pattern comparable to a normal step or fast locomotion. Finally, during a normal walk and when crossing a lawn full of small hidden obstacles, one can stumble. This results in a stair up or stair down f/e pattern but being labeled as normal walk. The other false detections occur intermittent and isolated and are due to transitions.
A further reduction in false classifications can be established in our opinion by taking in consideration that relevant activities are those with a repetitive nature. Hence, isolated misclassifications can then easily be removed when no repetitive nature in the activity is observed.
The proposed thresholds in the decision tree to distinguish between normal walking and fast locomotion are based on cadence. Cadence is used as this is a time-related gait parameter. The thresholds, 80 steps/minute to 150 steps/minute for normal walking and +150 steps/minute for fast locomotion, are relevant for young healthy subjects in real-life and ambulatory setting. By lowering the 80 steps/minute we see a decrease of the “no detection” for normal walk but an increase of false positives at the stairs ascend. This can be explained as follows. When a subject slows down, like strolling, this results in another knee f/e pattern. In this pattern, the expected peaks are not always obvious. Therefore, these peaks are filtered out by the smoothing step, which results in one maximum. This is therefore predicted as a stair ascend. As we want to detect walking with a certain level of physical exertion, a cut-off at 80 steps/minute is a good threshold. Next, young healthy subjects walking a predefined track reach already a cadence of 120 steps/minute during normal walking. Therefore, the proposed threshold of +120 steps/minute for fast locomotion by Tudor-Locke et al. [29] is too low in this study. We selected a cut-off of +150 steps/minute, which was also indicated by Egerton et al. [30]. This is a reasonable cut-off as we look to the confusion matrix. When lowering this threshold, there is a decrease in (a) correct classified normal walking and (b) fast locomotion. This can be explained by the fact that faster steps will be classified as fast locomotion but that they are labeled as normal walking. Also, there will be a shift in the confusion matrix in the row of stairs decent. The number of predicted normal walk strides will decrease and the number of predicted fast locomotion strides will increase. This is caused by the fact that a fast stair decent has a pattern comparable to a normal walk or a fast locomotion.
Compared to the current state-of-the art, we developed a measurement device (KB) where correct placement is not crucial. This implies that the patient can put on the measurement device by himself which is necessary for long-term monitoring. This is not feasible with devices like IDEEA as described by Huddleston et al. [22] or a combination of different MEMS as proposed by Kobashi et al. [17]. Furthermore, the KB does not make use of gyroscopes or other sensors [16, 23] where drift can cause errors in the output signals. Especially when the devices have to be used for long-term monitoring no complex or fusing algorithms are required to process the sensor data. Finally, this study is performed in the field where subjects were free to walk the track. We believe that this eliminates influences in measurements performed in a laboratory or under laboratory conditions. This novel technology will enable tailoring treatment and rehabilitation strategies to the needs for individuals including athletes in individual training schemes. An advantage of the inductive sensor used here is that it can easily be integrated in, or worn under, clothing which makes it comfortable to wear during rehabilitation, sports activities, and daily activities in ambulatory circumstances.
We presented an algorithm to measure quantitative parameters of physical activities and believe that we paved the road for the development of automatic detection algorithms of qualitative parameters. Qualitative parameters are, for instance, symmetry, stability, efficiency and regularity, which describes how physical activities are performed. In this paper we use peaks (local minima and maxima) to classify activities by just counting them. As future work we will investigate, for example, the ratio or the relative position of specific local minima and maxima in one activity cycle. This ratio or position can tell us something about the quality of the activity like stability or regularity.
Acknowledgments
The Biosensing project is supported by the “Pieken inde Delta” program and is granted by the Dutch Ministry of Economic Affairs, province of Limburg, province of Noord-Brabant, and the cooperation of the region Eindhoven.