Abstract
This paper proposes a novel approach to recognize activities based on sensor-placement feature selection. The method is designed to address a problem of multisensor fusion information of wearable sensors which are located in different positions of a human body. Precisely, the approach can extract the best feature set that characterizes each activity regarding a body-sensor location to recognize daily living activities. We firstly preprocess the raw data by utilizing a low-pass filter. After extracting various features, feature selection algorithms are applied separately on feature sets of each sensor to obtain the best feature set for each body position. Then, we investigate the correlation of the features in each set to optimize the feature set. Finally, a classifier is applied to an optimized feature set, which contains features from four body positions to classify thirteen activities. In experimental results, we obtain an overall accuracy of 99.13% by applying the proposed method to the benchmark dataset. The results show that we can reduce the computation time for the feature selection step and achieve a high accuracy rate by performing feature selection for the placement of each sensor. In addition, our proposed method can be used for a multiple-sensor configuration to classify activities of daily living. The method is also expected to deploy to an activity classification system-based big data platform since each sensor node only sends essential information characterizing itself to a cloud server.
1. Introduction
Daily living activity recognition is one of the most important research topics that can provide valuable information for various applications such as health monitoring, security, intelligent assistance in daily life, fall detection, and surveillance [1, 2]. It is obvious that the quantity and placement of wearable sensors on the human body are factors that affect the accuracy of activity classification [3–5]. Each kind of activity has different characteristics which are specific to the particular location on the human body.
The development of embedded sensor technology in wearable devices, that is, smartphone, smart band, and smart shoe, has featured a variety of daily activity recognition-related researches [6–8]. A major issue of these studies is that there is a difficulty in acquiring experimental results that carry all real-world applications while activities are conducted in a controlled environment [9]. Some recent publications [6, 7, 10], which utilized a single sensor module attached to different body positions, can achieve a high classification accuracy. However, a single sensor module-based platform cannot be sufficient to recognize complex daily living activities. In particular, characterizations of the activities may not be true using one sensor even though a high classification accuracy is achieved [11]. To overcome that problem, researchers have tended to attached many sensors to different parts of a human body [4, 11–14]. These approaches can recognize both dynamic and transitional activities. Furthermore, a multiple-sensor system, which contains up to three sensors, can provide a good recognition accuracy [11]. The fact remains that researchers have made efforts to deal with a fusion of multiple-sensor data. In addition, challenges of a variety of monitoring activities and precision of classification are directly impacted by the number and the placement of inertial sensors attached to the human body [15]. On-body device position was investigated to determine its influence on the performance of activity recognition [16, 17]. Figueira et al. [18] explored the sensor-position awareness of monitoring activities. Atallah et al. [19] discussed the sensor positions and significant features for each activity group according to the compendium of physical activities. However, a recognition system with many sensors can contain numerous redundant features which degrade the classification performance and may even confuse the classifier. Recent studies [19–22] have approved feature selection techniques as a solution to deal with the redundant features. There have been various activity recognition topologies designed with feature selection algorithms. Appropriately, a novel scheme based on feature selection is introduced to achieve a high activity classification performance.
In this paper, we propose a new scheme which is evaluated on the DaLiAc dataset for human daily living activity recognition based on sensor-placement feature selection. We first apply a low-pass filter to the raw data. The filtered data are segmented by a 10-second sliding window with no overlap to obtain a sample. Then, we extract valuable features from the filtered data. Feature selection techniques are applied separately on each sensor location to obtain the best feature set for each body position. We optimize the feature set by investigating the correlation of the features. Finally, a -nearest neighbor algorithm is applied to an optimized feature set containing features from four body positions to classify thirteen activities. The key of our framework lies in the feature selection layer. In this work, we implement feature selection on an individual set of sensor features. This approach optimizes the features for specific sensors, and therefore, facilitates correct selection of the features specific to each activity. The contributions of the paper are as follows: (i)It proposes a new method of daily living activity recognition that achieves a higher accuracy rate in a benchmark dataset in comparison with recent publications(ii)It discovers sensor positions that most affects the recognition rate in a benchmark dataset(iii)It explores meaningful features by utilizing three feature selection methods, that is, Laplacian score, SFFS, and Relief-F, as well as correlation-based feature optimization for four sensor locations in the ankle, chest, hip, and wrist
The rest of the paper is organized as follows. In Section 2, we present some related works on the recognition of daily living activities with one or more sensors. The proposed method is described in Section 3. Furthermore, we go into the details of data preprocessing, feature extraction, future selection, feature optimization, choice of classifier, and evaluation criteria. All comparisons of the selected set of features and the performance of the proposed method will be shown in Section 4. Finally, Section 5 presents the conclusion of this paper and future works.
2. Related Works
Many authors have reported activity recognition issues when using one sensor to monitor activities of daily life [6, 7, 10]. In our previous work [6], a multiclass support vector machine (SVM) algorithm demonstrated 97.3% accuracy when we used one module smart band with a triaxial acceleration stuck on the wrist to collect data during the performance of five wrist activities: texting, calling, hand in pocket, hand swinging, and suitcase carrying. Nguyen et al. [7] proposed three novel plantar-pressure sensor-based features to classify five ambulatory activities using smart insoles. By implementing a -nearest neighbor classifier, these walking activities can be recognized with an accuracy rate of 97.84% at the sixth step. Zheng [10] presented a hierarchical framework based on the least squares support vector machine and naive Bayes algorithm to classify ten activities using an inertial module attached on the hip. Their implemented method, which recognized activities regarding four groups (bidimensional walking, tridimensional walking, plane motion, and static activities), obtained a high accuracy rate of 95%. These studies can achieve high activity classification rates using a small size of feature vectors and simple classifiers. However, this approach, which used data collected from one sensor location, only attained the best recognition performance to predict activities that relate directly to the sensor’s location. This limits the number of activities for consideration.
To address the drawback of a single-sensor configuration, various studies have utilized multiple inertial sensors attached to different body locations to classify human activities [4, 9, 11–14]. Sztyler et al. [9] improved an activity recognition performance by being aware of the device position. Their method was implemented on a dataset of eight activities collected from seven on-body locations. In the experimental results, they obtained an F-measure of 89%. In addition, an awakening of sensor locations can enhance the performance of activity recognition compared with subject-specific and cross-subject approaches. Khan et al. [11] presented a cloud-based system to recognize thirteen activities using the WARD dataset [23]. Effects of on-body sensor position were explored by implementing three classifiers for each activity. Their analysis showed that activity recognition is significantly dependent on the positions of on-body devices. Furthermore, utilizing up to three sensor modules is needed to recognize more dynamic and transient activities. Schuldhaus et al. [12] proposed a decision level fusion-based method to classify seven daily living activities. Four sensor nodes, which were located on the chest, hip, wrist, and ankle, separately performed activity classification. Then, an activity was predicted by majority voting of each classifier decision of the sensor nodes. The designed system achieved an overall recognition error rate of 6.1%. Baños et al. [13] presented a metaclassifier framework which included an insertion and rejection algorithm of weights for base (classifier) and source (activity) levels, respectively. A voting algorithm is designed to fuse the decisions of individual classifiers to predict activities. To compare with other fusion models, this scheme can deal with misclassification caused by a failure of one or more sensors. The method which was evaluated on the REALDISP benchmark dataset [24] can improve the recognition performance compared with a traditional feature fusion, hierarchical-weighted classifier [25] approach to classify thirty-three activities. Gao et al. [14] developed a framework consisting of sensor selection and hierarchical classifier module to address the dynamic multisensor fusion problem. They investigated the recognition performance and energy consumption of the method on a dataset of eight activities. The experimental results demonstrated that a system can save 37.5% of the energy consumption and reduce the classification rate by 2.8% compared with an original four-sensor classifier approach. Bao and Intille [4] used features extracted from data collected using five biaxial accelerometers attached to the left elbow, right wrist, hip, left thigh, and right ankle of twenty different subjects to recognize twenty different activities. The overall accuracy using a decision tree was 84% without the utilization of a feature selection technique. The main advantage of multiple sensors is that they can recognize many daily living activities with high precision. Moreover, it is more suitable to recognize complex activities, that is, dynamic and transitional activities [26]. However, the main issue of this configuration is an optimal sensor combination to achieve a high classification performance and robustness [25]. In addition, the multiple-sensor systems can become uncomfortable for users who wear them for a long time [13, 25].
The wearable sensor community recently investigated feature selection techniques which demonstrated an efficient solution to deal with the problem of redundant features and computational expense [27] in recognizing human activities [19–22]. Feature selection algorithms were implemented in many ways, that is, performing on a total set of features extracted from various activities, applying to each activity group, and undertaking each activity. Atallah et al. [19] employed six wearable accelerometers positioned at six different body locations in the ear, chest, wrist, ankle, knee, arm, and waist. They used a filter method-based feature selection technique, that is, Relief [28], Simba [29], and minimum redundancy maximum relevance (MRMR) [30] to evaluate and rank the features. The -nearest neighbor with and and the Bayesian classifier were used to classify fifteen daily activities which were combined into five groups of activities labeled as very low level, low level, medium level, high level, and transitional activities. The performance of both algorithms was similar at approximately 82%. Zhang and Sawchuck [20] acquired data using a 6-DOF inertial measurement unit (IMU), which integrated a triaxial accelerometer, gyroscope, and magnetometer. This study utilized three selection techniques, that is, Relief [28], a wrapper method based on a single feature (SFC) [31], and a wrapper method based on sequential for selection (SFS) [32] to select the most important features. The SVM algorithm was then deployed to recognize twelve activities including walking, forward, walking left, walking right, going upstairs, going downstairs, jumping up, running, standing, and sitting. The accuracy of the single-layer classifier was up to 90%. Pirttikangas et al. [21] configured four triaxial accelerometers attached to the right wrist, left wrist, right thigh, and neck and one heart rate sensor attached to the right wrist of thirteen different users. The collected data from 17 different activities were characterized into five features, that is, acceleration mean, standard deviation, correlation, mean crossing, and heart rate mean. The forward-backward search was utilized to select a subset of the best features. The author used the -nearest neighbor as a classifier and the algorithm achieved 90.61% activity recognition accuracy. Huynh and Schiele [22] showed that the performance can be improved by implementing feature selection techniques on individual features for each activity such as walking, standing, jogging, skipping, hopping, and riding a bus. Data was collected using an integrated sensor board containing sensors for 3D acceleration, audio, temperature, IR/visible/high-frequency light, humidity, barometric pressure, and a digital compass. The board was placed on the shoulder strap of a backpack and the subjects were required to wear the backpack while performing the measured activities. The author applied cluster precision to evaluate the features and the best accuracy of recognition was about 90% when the best features and window length of jogging and walking were considered. For the other activities, the recognition accuracy was about 85%. The authors applied feature selection to individual activities, but the results did not show improved efficiency when the best set of features for all activities was combined into one global set as the input for the classifier.
Some state-of-art methods evaluated on the DaLiAc dataset [33], which obtained acceptable classification accuracy rates to recognize thirteen activities, can be found in [33, 34]. Leutheuser et al. [33] achieved an average classification rate of 85.8% utilizing a hierarchical multisensor-based activity recognition system. Furthermore, their proposed method can distinguish between complex daily living activities, that is, treadmill running and rope jumping. Zdravevski et al. [34] proposed a systematic feature engineering method that uses a two-phase feature selection layer. The method achieved an accuracy rate of 91.2% with 60 selected features to classify thirteen activities.
3. Activity Recognition Algorithm for Daily Living Activities
In this section, we first present a comparison between the proposed and conventional algorithms. Then, data preprocessing, feature extraction, selection, optimization, choice of classifier, and evaluation criteria are discussed.
3.1. Feature Selection Based on Body-Sensor Placement
Figures 1(b) and 1(a) respectively illustrate the proposed and standard approaches to recognize human activity using wearable devices. In the proposed architecture, the feature selection layer is different from that of the standard approach. In this layer, the proposed method separately performs feature selection on each sensor position, whereas the standard method considers all extracted features at once. The main advantage of the proposed method is that each sensor has a different set of features which is better for depicting the activities with regard to the position of the sensors. Moreover, the computation requirement of feature selection is significantly decreased compared with the standard approach. In addition, the standard approach deployed in most studies [19, 21, 33] achieves medium classification accuracy (below 95%). The disadvantage of the standard approach is that it becomes computationally expensive as the number of features increases. We also perform optimization based on the Pearson correlation coefficient to take high-correlation features into account in the proposed method.
(a)
(b)
3.2. Data Preprocessing
In this study, inertial data were uniformly resampled with a sampling frequency of 200 Hz. We cut out a segment of the inertial signal using a 10 s sliding window with no overlap to obtain the activity samples. A low-pass filter was then applied to these samples to remove noise. Table 1 shows the distribution of the human activity samples. Examples of the acceleration and gyroscope signals for sitting are shown in Figure 2.
(a) x-axis acceleration
(b) y-axis acceleration
(c) z-axis acceleration
(d) x-axis angular rate
(e) y-axis angular rate
(f) z-axis angular rate
3.3. Feature Extraction
Feature extraction transforms high-dimensional raw data into a meaningful representation data that can depict the characteristics of different activities. This technique enables machine learning algorithms to deal with high-dimensional data. We obtain a large set of features calculated using the filtered data for each activity. To perform feature extraction, we implement eight features on smaller windows of 2 s data, that is, signal magnitude area, intensity of movement, mean trend, windowed mean difference, variance trend, windowed variance difference, standard deviation trend, and windowed standard deviation difference. In addition, the 10 s long inertial signals are divided into 5 windows with no overlap. The remaining features were extracted directly without breaking the samples into smaller windows.
The details of the feature are shown in Table 2.
The standard deviation trend and windowed standard deviation have different features, which are computed similar to the mean trend and windowed mean difference, and are given below: (i)Standard deviation trend(ii)Windowed standard deviation difference
The extracted features are not distributed normally and are sparse which can affect the performance of estimators. Therefore, feature normalization is a common requirement for many machine learning algorithms. In our study, we normalized all the features to zero mean and unit variance. where and are the mean and standard deviation of a particular feature, respectively.
3.4. Feature Selection
We obtain a total of 536 features including 192 statistical features, 120 magnitude-based features, 24 correlation-based features, 56 frequency-domain features, and 144 trend-variance features after performing feature extraction. In order to improve the classification accuracy and reduce the expense calculation time, the redundant (features are closely related) and irrelevant features should be removed [27, 35]. This task can be performed with feature selection techniques. There are three common groups of feature selections: filter method, wrapper method, and embedded method [36]. Filter methods which are computationally efficient select features that are independent of any machine learning algorithms. On the contrary, wrapper methods evaluate the quality of the selected features based on the performance of a predefined learning algorithm. Embedded methods embed the feature selection with model learning. In this paper, we use 2 filter methods and 1 wrapper method, that is, Relief-F, Laplacian Score, and Sequential Forward Floating Selection (SFFS) to rank the features for each sensor location. These methods are briefly described below.
Relief-F [37] is a popular filter method and is extended from the original feature selection method Relief [28]. This method finds the nearest instances from different classes ( nearest hits/misses) using the Manhattan norm instead of only one nearest hit and miss when using the Euclidean norm and then updates the weight of each feature. Relief-F can also handle incomplete data and the multiclass problem, which are problems in the Relief algorithm. If feature is considered, then the weight will be updated following [37]: where calculates the difference in feature value between two instances and in the near hit class (same class as ), and calculates the difference in features between two instances and in the near miss class (different class from ).
Laplacian Score algorithm is another feature selection technique based on the filter method [38]. Suppose we have data points, , and the th node corresponds to . In order to select the significance features, first, the Laplacian score algorithm constructs a -nearest neighbor graph with nodes and defines an edge in the graph for the node if another node is one of its -nearest neighbors. Second, it calculates the weight matrix with the element ( is constant) if any two nodes and are connected. In [38] the Laplacian score of the th feature was defined as where denotes the vector of samples for the th feature. is a Laplacian matrix defined as [39] and is a diagonal matrix defined as .
The third feature selection technique used in the paper is Sequential Forward Floating Selection [40]. This technique is extended from the Sequential Forward Selection (SFS) method. Let us assume that we have a set of dimensional features . The SFS will output a subset of dimensional features with and assessed by a criterion function with . First, SFS takes all dimensional features from the original feature set as its input and will start adding the first feature which has the highest significance subject to
From the initial subset , SFS extends more features according to
Then, the new subset of features becomes
The disadvantage of SFS is that once a feature is selected, it cannot be removed from the subset. SFFS can solve this problem by checking and removing the least significant feature in the subset for each iteration, that is
This procedure is repeated until the number of selected features equals the desired number of features or .
After ranking the features for each sensor location, we use -NN to consider various features from high to low rank based on the model accuracy of each feature selection technique. Algorithm 1 presents the model accuracy procedure used to select the important features. A leave-one-out cross-validation and a -NN classifier are utilized in the algorithm.
| ||||||||||||||||||||||||||||||||||||||||||||
3.5. Feature Set Optimization
Relief-F and Laplacian Score select features by evaluating the capabilities of the features in preserving sample similarity. Therefore, they are unable to handle feature redundancy, which may have a high correlation with each other [41]. In addition, SFFS which cannot eliminate redundant features generated in the searching process, does not consider the feature correlation [42]. To address the redundancy caused by high-correlation features, we use the Pearson correlation coefficient [4, 43] with a threshold equal to 1 in our study. Let us assume that we have two data segments and . Then, the Pearson correlation is defined as
3.6. Classifier
-nearest neighbor (-NN) and support vector machine (SVM) are popular supervised classification algorithms in machine learning problems. SVM is robust and delivers a unique solution because of convex optimization. However, this technique is more computationally expensive when dealing with high-dimensional data [44, 45]. In contrast, -NN, which is a simple and robust algorithm, does not require a learning process [15, 46]. It uses the principle of similarity (Euclidean distance) between the training set and the new observation to be classified. Therefore, it is easy to implement and less computation intensive in comparison to SVM. The problem with -NN is that it is very sensitive to redundant features [45]. In this study, we use the -NN classifier to choose the number of features and the classification.
3.7. Evaluation Criteria
We evaluate the performance of our proposed method by utilizing four metrics, that is, accuracy, precision, recall, and F1-score. The accuracy can be expressed for binary classification as follows: where true positive (TP) means that an instance labeled as positive is predicted as positive. A true negative (TN) is defined when the algorithm predicts a negative result for a label that is indeed negative. When the label is negative but is predicted as positive, the prediction is defined as a false positive (FP). The reverse situation for a false positive is called a false negative (FN).
F1-score which is the combination of two metrics, that is, the precision and recall, is defined as follows:
4. Experimental Results
In this section, a brief description of the used dataset is given. Then, we present the results of the feature selection and feature optimization procedure. The performance of our proposed method and a comparison with the reference papers are also presented. In addition, three metrics, that is, precision, recall, and F1-score are utilized to evaluate the proposed method’s performance. The comparison highlights the performance of the different methods in terms of the F1-score.
4.1. DaLiAc Dataset
In this study, we use the DaLiAc (Daily Living Activities) dataset, which is a benchmark dataset for the classification of daily life activities based on inertial data as described in [33]. Nineteen healthy subjects (8 women and 11 men, age 26 ± 8 years, height 177 ± 11 cm, weight 75.2 ± 14.2 kg) performed thirteen activities, that is, sitting (labeled as SI), lying down (LY), standing (ST), washing dishes (WD), vacuuming (VC), sweeping (SW), walking (WK), ascending stairs (AS), descending stairs (DS), treadmill running (RU), bicycling on an ergometer (50 W) (BC50), bicycling on an ergometer (100 W) (BC100), and rope jumping (RJ). Inertial data were collected from four Shimmer sensor nodes [26] placed on the right hip, chest, right wrist, and left ankle. Figure 3 illustrates the positions and coordinates of each sensor.
Each sensor node consisted of a 3D accelerometer and a 3D gyroscope. The range of the accelerometer was ±6g. The range of the gyroscopes was ±500 deg/s for the sensor nodes at the wrist, chest, and hip and ±2000 deg/s for the sensor node on the ankle. The sampling rate was set to 204.8 Hz.
4.2. Results of Feature Selection and Feature Optimization
The experimental results of feature selection are given in Figure 4 and Table 3. Figure 4 illustrates the classification accuracy according to the rank of the selected features for the four sensor locations. In detail, the obtained results are summarized in Table 3.
(a)
(b)
(c)
(d)
In Table 4, it is clear that the Laplacian Score outperforms SFFS and Relief-F in terms of the number of the selected features and accuracy. Among the sensor positions, the ankle and hip, which have the same number of selected features (29 features), give a high overall accuracy, that is, 96.85% and 96.91%, respectively. The accuracy rates of the three feature selection methods are very close, but the number of features selected by Laplacian Score is the lowest number, that is, 36 features in comparison with 87 features (SFFS) and 77 features (Relief-F). Considering the wrist sensor, Laplacian Score achieves an accuracy rate of 95.24% with 44 features, whereas SFFS and Relief-F attain classification rates of 91.60% (124 features) and 92.75% (73 features), respectively.
Figure 5 presents in detail the distribution of selected features for each sensor position after optimization. Wrist location contributes most of features (32%) for activity classification which is shown in Figure 5(a). According to Figure 5(b), the mean absolute deviation (MAD), range (), correlation (), and standard deviation () features for one or more axes overwhelm four sensor positions, that is, the hip (71.43% in total), ankle (66.66%), chest (58.82%), and wrist (52.38%).
(a)
(b)
4.3. Performance of the Proposed Method and a Comparison with Reference Methods
Table 5 presents the confusion matrix for the recognition of thirteen daily living activities.
Table 3 shows the performance evaluation of our proposed method in detail. The proposed method recognizes washing dishes, descending stairs, treadmill running, and rope running with the highest accuracy rate of 100%. The method also achieves high accuracy for walking (99.62%), lying (99.47%), and ascending stairs (99.47%). Differences in intensity cause confusion for the algorithm when distinguishing bicycling on an ergometer (100 W) (98.93%) and bicycling on an ergometer (50 W) (98.72%). The confusion matrix in Table 5 clearly shows that these activities are misclassified as each other. The accuracy for sweeping (98.27%) and vacuuming (98.06%) are quite similar. Sitting and standing, which have the same perspective of orientation in gravity, are misclassified by 2.97% and 3.16%, respectively.
Table 6 gives a comparison of the proposed method with two reference papers, that is, Leutheuser et al. [33] and Zdravevski et al. [34]. As shown in Table 6, our proposed method compares well against the reference papers in terms of recognition of sitting, standing, washing dishes, walking, ascending stairs, descending stairs, treadmill running, and rope jumping. Overall, our proposed method achieved an accuracy rate of 99.13% in classifying 13 daily living activities, whereas the two reference methods of Leutheuser et al. and Zdravevski et al. achieved accuracy rates of 89.00% and 93.00%, respectively.
5. Conclusion
We proposed a novel approach that applies a feature selection algorithm on different sensor locations to recognize thirteen daily living activities using the DaLiAc benchmark dataset. To perform the recognition, we extracted features from a preprocessed dataset. Then, the features were ranked according to feature importance using a feature selection technique. In addition, we performed this process separately for each sensor-location dataset. Before feeding the features to a classifier, we optimized the selected features by using the Pearson correlation coefficient with a threshold equal to 1. We evaluated the performance of our proposed method with three metrics, that is, precision, recall, and F1-score. The results showed that the proposed method achieved a high precision rate of 100% for five activities, that is, lying, walking, ascending stairs, treadmill running, and rope jumping. The lowest precision rate was 95.73% in recognizing vacuuming activity. Furthermore, we compared the F1-score metric with the two reference papers. Overall, our proposed method outperformed the two reference methods with an accuracy rate of 99.13% in classifying thirteen activities. In contrast, Leutheuser et al. and Zdravevski et al. only achieved accuracy rates of 89.00% and 93.00%, respectively. The results are promising and show that the proposed method can be useful for a multiple-sensor configuration in human activity recognition.
In the future, we plan to improve the accuracy rate and reduce the feature dimension for classifying activities of daily living. In addition, we will extend our work by taking into consideration the feature correlation and the feature rank among the sensor positions.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare no conflict of interests.
Acknowledgments
This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2015R1D1A1A01060917). This research was also supported by the National Research Foundation of Korea (NRF) funded by the Korean Government (MSIP) (NRF-2016R1A5A1012966).