Abstract
As the world’s population of senior citizens continues to grow, the burden on the professionals who care for them (carers) is also increasing. In nursing homes, carers often write daily reports to improve the resident’s quality of life. However, since each carer needs to simultaneously care for multiple residents, they have difficulty thoroughly recording the activities of residents. In this paper, we address this problem by proposing an automatic daily report generation system that monitors the activities of nursing home residents. The proposed system estimates the multiple locations (areas) at which residents are situated with a BLE beacon, using a variety of methods to analyze the RSSI values of BLE signals, and recognizes the activity of each resident from the estimated area information. The information of the estimated activity of residents is stored in a server with timestamps, and the server automatically generates daily reports based on them. To show the effectiveness of the proposed system, we conducted an experiment for five days with four participants in cooperation with an actual nursing home. We determined the proposed system’s effectiveness with the following four evaluations: (1) comparison of performance of different machine-learning algorithms, (2) comparison of smoothing methods, (3) comparison of time windows, and (4) evaluation of generated daily reports. Our evaluations show the most effective combination pattern among 156 patterns to accurately generate daily reports. We conclude that the proposed system has high effectiveness, high usability, and high flexibility.
1. Introduction
In recent years, due to the astonishing progress of medical technology, human life expectancy has shown a consistent tendency to increase [1]. The senior populations continue to increase year by year especially in developed countries. Japan has one of the world’s highest life-expectancy rates, and a quarter of its people are over 65. Such aging societies pose many problems for society. One critical problem is the increase of the burden on carers in nursing homes, who are responsible for multiple elderly people, that is, nursing home residents.
According to an interview with the owner of an actual nursing home, Ikoi-no-ie 26 (Ikoi-no-ie 26 (Japanese website): http://www.lifecarejp.com), the burden on the carers at such facilities continues to increase, and many carers leave the industry quickly due to the harshness of their work. Many nursing homes suffer from a chronic shortage of carers. In nursing homes, carers perform a wide variety of tasks, including health checks, rehabilitation support, food preparation, restroom support, and conversation. As illustrated in Figure 1, they also have to write daily reports (that record some primary resident activities) that are used by the government to determine funding for nursing homes. Under aging societies, the situation degrades the service quality, since the carers must prepare daily reports on multiple residents, which disrupt other tasks.
For example, assume a situation where a carer is looking after two residents. One resident is doing her rehabilitation work. The carer has to fill out a daily report on her rehabilitation progress while simultaneously supporting that rehabilitation activity. The other resident is going to the restroom. The carer has to stop filling out the daily reports and support him, which means that the carer cannot accurately report on the rehabilitation. Hence, recording the activities of multiresidents is very hard and writing daily reports places a heavy burden on carers [2]. This burden is a serious problem that impairs the ability of nursing homes to provide safe, high-quality service and must be solved.
To help alleviate this problem, we aimed to determine how ICT/IoT technology can be used to support the recording work of carers in nursing homes. We set the following goals for the desired system: (1) creating a multiresident activity monitoring system and (2) automatically generating daily reports for each resident.
In this paper, in order to achieve these goals, we propose a system that uses BLE beacons, coupled with machine learning using the RSSI values of the BLE signals as the feature quantity, to recognize the location of each resident in the nursing home. The estimated location of each resident is then used to identify the activity they are engaged in. The information of the recognized activity is stored in a server with timestamps, and the server automatically generates daily reports based on them.
Our work makes the following contributions: (i)First, we provide an architecture called a Movable-Beacon and Fixed-Scanner (MBFS), which is our proposed system’s key concept. In this architecture, a BLE beacon is carried by residents and scanners are installed on the target area on the environment side. In existing studies [3–5], many beacons are placed in the environment and users have smartphones for estimating their current area. We call this architecture a Fixed-Beacon and Movable-Scanner (FBMS). However, using FBMS in nursing homes is unrealistic because of low smartphone penetration among residents and the burden caused by forcing them to always carry smartphones. In this paper, we argue that the presented architecture decreases the burden on residents.(ii)Second, we describe the system based on the MBFS architecture that can monitor the activity of nursing home residents and automatically generate daily reports corresponding to each of them. Specifically, we show that our system is a simple composition which has only four components—a BLE beacon, a scanner, a server, and a web application—and a very useful design that can easily check the information of each resident by carers.(iii)Third, we develop a BLE beacon with an accelerometer. This BLE beacon consists of the following three components: an accelerometer, a microprocessor with a BLE module, and a battery. A BLE beacon, which is attached to the waists of the residents, has a unique development and resembles a name tag. Therefore, its burden on residents is small. In Section 6, we describe how our developed BLE beacon will be used for rehabilitation support in the future.(iv)Fourth, we show the effectiveness of our proposed system through a five-day experiment with four participants in cooperation with a nursing home called Ikoi-no-ie 26. We evaluated the effectiveness of our proposed system and expand our previous work [6], by conducting the following four evaluations: (1) comparison of machine-learning algorithms for classification, (2) comparison of smoothing methods, (3) comparison of time windows, and (4) evaluation of generated daily reports. Our evaluations provide the most effective combination pattern in the proposed system to accurately generate daily reports.(v)Finally, we show that the proposed system clearly has high effectiveness and conclude that the research goals set in this study are sufficiently achieved based on the experimental results.
The rest of this paper is organized as follows. The preliminary work is provided in Section 2, and Section 3 presents our automatic daily report generation system. In Section 4, we describe the experimental environments and evaluation methods. The evaluation results are described in Section 5, and a discussion of the usability and flexibility of our system is provided in Section 6. Section 7 reviews existing work related to this paper. Finally, Section 8 concludes this paper.
2. Preliminary Work
The goals for our study were to (1) create a multiresident activity monitoring system and (2) automatically generate daily reports for each resident. To achieve them, we conducted several preliminary works under the cooperation of Ikoi-no-ie 26. In this section, we explain the basic working environment in nursing homes and their motivation, which is crucial to conduct our study. Then, we describe previous works [2, 7–10] and explain the requirements for the development of our proposed system, as described in Section 3.
2.1. Basic Knowledge and Motivation
In nursing homes, carers perform many kinds of tasks while simultaneously taking care of several residents. For example, they are involved in health checks, rehabilitation support, food preparation, restroom support, conversations, and so on. Recording daily reports on the residents is one of their most important tasks. The daily report is the nursing report that monitors the activity of residents and records their activity in detail. Such reports are used for the following three main purposes: (1) information sharing among carers, (2) deciding subsequent rehabilitation plans, and (3) communication between residents’ families and carers. Hence, daily reports are critical for both residents and carers.
However, from interviewing the nursing home’s owner, we found that recording the daily reports of the resident activities is onerous. This problem has also been cited in several papers. Miwa et al. [11] reported that recording work accounts for approximately 25% of the daily work of carers. Inoue et al. [12] described documentation as one of the most time-consuming duties for nursing reports. The recording and documentation responsibilities of daily reports are the biggest barriers to improving the quality of service in nursing homes.
Based on these reasons, we believe that solving the above problems is very important and academically and socially valuable in aging societies. Thus, to lighten the heavy burden of recording the daily reports, we focused on the development of a system that monitors several nursing home residents and automatically generates daily reports corresponding to each of them.
2.2. Previous Work
To realize the above system, it is necessary to monitor a resident in the nursing home. First, as a prototype system, we presented a novel system [7, 8] in which users have BLE beacons and installed scanners on the environment side. In this system, we divided a nursing home into several areas and installed scanners in each one. BLE beacons were attached all day to the clothing of the resident, and scanners received BLE beacon signals. Then by comparing the calculated RSSI values of the BLE signals at each scanner, this system estimated the area with the strongest RSSI value as the resident’s current area. As the result of an experiment we conducted at Ikoi-no-ie 26, the existing area of one resident was estimated at a precision of 59.5%. However, the system faced a serious problem. Its area estimation accuracy was too low because the RSSI values dramatically changed based on the installation position of the scanners or the facility’s environment. In addition, when installing scanners, we had to consider particular problems of nursing homes, such as residents who are sensitive to changes in their surrounding environment and the positions of the power supply. We needed to camouflage our installed equipment so that residents could not see the scanner positions (Figure 2). Therefore, we needed to place our system at a different height to the installation positions of the scanners or hide them behind training equipment. However, due to the difference in the installation position of scanners, there was the problem where the scanners of neighboring area became nearer than the scanner of the area where the resident was located. Hence, we found that estimating the area where the resident is located is difficult by just simply comparing RSSI values.
(a) Scanner installation work
(b) Camouflage
Next, to solve the problem in the above system, we estimated the resident’s current area by machine learning [2, 9, 10] with an estimation method that considers the influence of the height difference of each scanner in each area in the nursing home. In our experiment at a nursing home that shows the system’s effectiveness, the existing area of one resident is evaluated by the 10-cross-validation method, and the -measure was 80.6%. In addition, this paper showed that the system can relatively accurately generate daily reports with the estimation results of the existing area. However, we did not conduct any experiments with multiple residents in these papers. Estimating the existing area of multiple residents is important, but actually our experiments were only for one resident. This reason is that it is likely to conflict RSSI signals sent from multiple residents at some BLE scanners and the activity recognition accuracy would be worsened if this system is used as is. Also, these papers pointed out that small erroneous estimations in seconds were generated from the evaluation results. Therefore, we indicated that the accuracy of activity recognition can be further improved by using a smoothing method which is able to eliminate the erroneous estimation.
2.3. System Requirements
Through our previous work and interviews with the nursing home’s owner, we identified some problems. Since the nursing home has many residents, our system needs to simultaneously monitor multiple residents. However, our previous work did not consider this. Moreover, to monitor the location of residents, we assumed that they were carrying smartphones that identify their locations. However, the interview with Ikoi-no-ie 26 revealed residents disliked this responsibility since the devices are heavy and bothersome. Therefore, a system using a BLE beacon is more desirable, as in previous work. In addition, a system that does not use cameras or microphones is required in nursing homes because such devices intrude on resident’s privacy (We define “privacy intrusion” as “unnecessary and excessive surveillance,” e.g., cameras or microphones that monitor residents.)
Thus, we need to develop a system that protects the privacy of residents and develop a low-cost system since most nursing homes are facing budget constraints, that is, insufficient funds to purchase cameras or smartphones for each resident. In addition, in the nursing home, a system is required that can immediately incorporate new residents because they often change. Based on the above discussion, we define the following system requirements: Req. 1:it should enable tracking of multiple residents.Req. 2:it should be a device that places small burden on residents who are carrying it.Req. 3:it should generally protect residents’ privacy.Req. 4:it should be low-cost.Req. 5:it should be available for new residents without data collection.
Thus, in this paper, we monitored the activities of multiple residents using a system that satisfies the above five requirements and automatically generated daily reports using the information of each resident.
3. Proposed System
Based on the discussion in Section 2.3, we propose an automatic daily report generation system that monitors the activity of nursing home residents. Our proposed system estimates the present area of multiple residents utilizing machine learning with the RSSI values of BLE signals as the feature quantity and recognizes the activity of each resident from the estimated area information. The information of the recognized activity of each resident is stored in the server with timestamps, and then the server automatically generates daily reports on each resident based on them. In this section, we outline our proposed system, explain the MBFS (Movable-Beacon and Fixed-Scanner) architecture, which is our proposed system’s key concept, and describe the application design that is based on the MBFS architecture. Finally, we explain the activity recognition process.
3.1. Outline
In nursing homes, the target activities are area-dependent activities (Figure 3); for example, if a resident is in the restroom area, we can recognize her activity in the restroom. When a resident is in the rehabilitation area, her activity is doing rehabilitation. We installed fixed scanners in the target areas where activities occur that we want to monitor, and residents carry BLE beacons. The system recognizes specific activities by utilizing machine learning with the RSSI values of the BLE signal observed by each scanner as features. In our proposed system, the FBMS (Fixed-Beacon and Movable-Scanner) architecture cannot be adopted in our target environment, since the residents never carry smartphones in the nursing home. Therefore, our proposed system adopts the MBFS architecture through which BLE beacons are carried by residents and scanners are installed in the target area of the environment side. Here, we summarize the following aspects and characteristics of our proposed system: (i)The proposed system can track the activities of multiple residents because it can distinguish among them by UUID, major value, and minor value included in the BLE signal.(ii)The proposed system is based on MBFS architecture. Therefore, its burden on residents is small since they just carry a BLE beacon.(iii)The proposed system has low privacy concerns because it does not use cameras or microphones. Our system just observes the signals of the BLE beacon.(iv)The proposed system is relatively inexpensive since its simple composition has only four components: a BLE beacon, a scanner, a server, and a web application.(v)The proposed system can incorporate new residents without data collection by simply adding a BLE beacon.
Hence, the proposed system satisfies requirements 1 to 5 mentioned in Section 2.3.
3.2. MBFS Architecture
In this paper, we explain an architecture called MBFS to solve the problem of FBMS architecture. Generally, residents never carry smartphones in the nursing home. However, many existing studies estimate their activities using smartphones. In these studies [3–5], many beacons are placed in the environment side and users have smartphones that recognize their activity. We call such architecture FBMS. In fact, when recognizing the activity of nursing home residents, a tag-less system is ideal because little special equipment must be carried. Thus, FBMS architecture, which requires that smartphones be carried, places a heavy burden on residents.
Figure 4 compares the MBFS and FBMS architectures. MBFS architecture is based on an idea that is opposite to FBMS architecture. In recent years, several research studies that resemble MBFS architecture have been conducted [13, 14]. In MBFS architecture, the user side carries a BLE beacon, and scanners are installed on the target areas of the environment side. Therefore, the burden on the residents is very light because they do not need to carry smartphones in the nursing home. BLE beacons are designed to minimize the burden on residents since they are embedded in the name tags that are constantly worn by the residents or are attached to positions that do not affect daily life, such as the waist. Hence, as a system to be introduced to nursing homes, MBFS architecture is more suitable than FBMS architecture that requires residents to carry smartphones as scanners.
3.3. System Design
In this subsection, we describe the design of the proposed system. Figure 5 shows its composition and data flow. The system is composed of four components: (1) a BLE beacon, (2) a scanner, (3) a server, and (4) a web application. The proposed system’s design is based on MBFS architecture. We explain each component below. (1)BLE beacon: the BLE beacon’s development is unique, relatively inexpensive, and downsized. Our developed BLE beacon is composed of an accelerometer, a microprocessor with a BLE module, and a battery. It can also send acceleration data in addition to the basic data of such BLE signals as UUID, major value, and minor value. The BLE beacon integrates these data and sends them to each scanner installed on the target areas at intervals of approximately five packets per second. Since our BLE beacon has an accelerometer, it will be used for rehabilitation support in the future. A more detailed explanation of BLE beacons is discussed in Section 6.(2)Scanner: we developed a scanner using a Raspberry Pi, equipped with a BLE dongle (its receiving sensitivity is approximately −100 to −50 dBm) for receiving signals from BLE beacons and a Wi-Fi dongle for sending data to the server. A scanner with Raspberry Pi is less expensive than using a smartphone as the scanner. When each scanner installed in the target areas receives data from the BLE beacons, the signal’s RSSI values of each BLE beacon are calculated and the data are sent to the server by Wi-Fi by adding the data of the installation area and a timestamp.(3)Server: the data sent from each scanner are first stored in the server’s database (DB). At this time, the server distinguishes each resident by UUID, major value, and minor value. The server estimates the present area of each resident by machine learning, based on the data in the DB. Then the server recognizes the activity of each nursing home resident based on the information of the estimated area of each resident and generates activity reports.(4)Web application: finally, the information on the activity reports of each resident is automatically generated in real time on the web application as daily reports for each resident.
The proposed system is very useful because carers can easily check the information of each resident anytime and anywhere in real time in the nursing home. But of course, this system must accurately recognize the activity of each resident. Thus, in Section 3.4, we describe the activity recognition process in detail.
3.4. Activity Recognition Process
In a small nursing home like Ikoi-no-ie 26, its interior construction is often an open space. Therefore, estimating the current area of each resident is complicated by the influence of the multipath and/or conflict of the BLE signals and the problem of the different height of the installation position of the scanners. In other words, recognizing the activity of each resident is very difficult. Hence, our most important technical challenge is accurately recognizing the activity of each resident. In this subsection, we explain the process of activity recognition in the proposed system. We estimate the present area of residents by utilizing machine learning and recognize the activity of residents based on the information of estimated area. We describe the activity recognition process in the following three phases: (1) data collection, (2) construction of classification model, and (3) smoothing. Each phase is described below.
3.4.1. Data Collection
In our proposed system, we use the RSSI values of BLE signals as features. RSSI values are collected based on the flow described in Section 3.3. Approximately five BLE signals are sent per second. However, BLE signals are not stable because their strength slightly changes depending on the environment. Unstable signals may influence the recognition of the activities of residents. Therefore, before utilizing machine learning, we need to process the data to reduce the influence of unstable signals. We minimize such influence by calculating the simple mean value of the RSSI values that are obtained per second.
Another problem is that packet loss occurs more frequently than expected because BLE signals are comparatively weak. If packet loss occurs, the system might not receive any BLE signals at all. The system cannot accurately recognize the activity of each resident when the RSSI values cannot be obtained. We need to seriously deal with this problem. Thus, we must complement the RSSI values that could not be received in advance by using dummy data, in order to accurately recognize the activity of each resident.
As shown in Figure 6, for the parts (i.e., the parts indicated by “–”) where RSSI values could not be obtained due to packet loss, the system complements for them by −100 dBm, which is the lower limit of the receiving sensitivity of a BLE dongle as dummy data. In contrast, for the parts where the RSSI values were obtained, the system calculates the simple mean value of the RSSI values that were obtained per second to reduce the influence of unstable signals. If more than one RSSI value is obtained per second, we do not compensate for the missing values using dummy data even if packet loss occurs. Since the dummy data have very low values compared to the other obtained RSSI values, when simple mean values are calculated using RSSI values including the dummy data, the values become incorrect. In this case, we do not compensate for the missing values using dummy data.
We also installed cameras in Ikoi-no-ie 26 to acquire the groundtruth data after gaining permission from the owner of Ikoi-no-ie 26, its residents, and their families. We manually labeled the collected data while viewing the recorded video. For labeling, we assigned numbers to each area in advance.
3.4.2. Construction of Classification Model
After the data collection, we constructed classification models based on the collected data. We used scikit-learn [15], which is a machine-learning library for constructing the classification models. This library constructs classification models based on a large number of machine-learning algorithms. We constructed classification models by adopting the following four machine-learning algorithms that are used in several kinds of studies [16–19]: (1) logistic regression, (2) support vector machine, (3) random forest, and (4) gradient boosting decision tree.
Logistic regression (LR) [20, 21] is a well-known statistical classification algorithm for predicting dichotomous dependent variables. In addition, we can classify multiple classes by setting multiple applied variables to provide quicker and more robust results than other classification algorithms.
The support vector machine (SVM) [22] is a typical pattern recognition and supervised learning algorithm that analyzes data used for classification and regression analysis. This algorithm’s characteristics include a maximum-margin hyperplane and a kernel trick. It has higher generalization ability among the learning algorithms of data classification.
Random forest (RF) [23] is a machine-learning algorithm based on ensemble learning. It uses multiple decision trees [24] as weak classifiers and obtains classification results by integrating the results from these weak classifiers. It has higher performance with shorter computation time for particular targets than other algorithms.
The gradient boosting decision tree (GBDT) [19, 25] is an ensemble algorithm that uses decision trees. Like other ensemble algorithms, it is built incrementally where each successive estimator reduces the previous model’s error. It takes longer to build a model than random forests because each tree has to be built based on the results of a prebuilt tree.
The comparison of the performance of these machine-learning algorithms in our study is described in Section 5.
3.4.3. Smoothing
Figure 7 shows an example of an activity classification result, called a daily report in this paper. This figure shows the result of displaying a resident’s recognized activities for every second in a time-series order. From this result, we identified many misclassifications, which were caused by the influences of the multipaths and/or conflicts of BLE signals and the problem of the different height of the installation position of the scanners. In this way, even with machine learning, the influence of many unstable signals remains. Therefore, to accurately recognize the activity of residents, we need to reduce the influences of unstable signals even after machine learning. In this paper, we propose the following two smoothing methods: (1) by most frequent value (MFV) and (2) by machine learning.
First, we describe smoothing by the most frequent value (MFV). Figure 8 shows an example of smoothing by the most frequent value (time window is set as odd seconds). We first set a time window and use a range that performs smoothing from the classification result. If the time window is set as odd seconds, we need to give the margin of (time window/2) in the front and rear of the smoothing range. At this time, the decimal point is suppressed. If the time window is set as even seconds, we need to give a margin of {(time window/2)−1} in front of the smoothing range and a margin of (time window/2) in its rear. If the time window is an even number, the way of giving the margin may be reversed. This margin is necessary for counting the most frequent value in the start and end points of the smoothing range, as shown in Figure 8. Even though the data within this margin cannot be displayed on the daily reports, it is not a problem because at most about several tens of seconds of observation data are affected for one day.
Then, we count the most frequent value of the area numbers within the time window. The most frequent value is decided as the classification activity within the time window. Then, we replace the part surrounded by the red squares in Figure 8, which is the middle part of the time window, with the identified classification activity. If there are two or more most frequent values at this time, we determine the classification activity within the time window based on high-priority activity that is set in advance. We defined high-priority activity as activities that we want to reliably recognize in the daily reports, especially the toilet and the bath that contain critical information. To prioritize activities other than toilet and bath, we also decided them comprehensively based on information on the staying time in each area of the residents and advice from the nursing home’s owner. We did smoothing while sliding the time window one by one from the start to the end point of the smoothing range.
Second, we describe smoothing by machine learning. This method smoothens by machine learning using the classification pattern (i.e., the part surrounded by the blue dashed line in Figure 8, e.g., “11514,” and “15145”) of the activities within the time window as a feature quantity. At this time, the groundtruth data uses the actual activity of a resident. We used three of the four machine-learning algorithms (excluding SVM) described in Section 3.4.2 for creating classification models. For a real system, since tuning based on each nursing home is required, we believe that SVM, which requires learning time, is undesirable in the smoothing process. In this method, in the same smoothing procedure as a smoothing method based on the most frequent value, smoothing is performed using constructed classification models.
A comparison of two kinds of smoothing methods (four patterns) is described in Section 5.
4. Experiment
In this section, we describe our experiment using the proposed system in an actual nursing home. First, we outline our experiment and explain its environment. Finally, we explain the evaluation method.
4.1. Outline of Experiment
To show the effectiveness of our proposed system, we conducted an experiment at Ikoi-no-ie 26 and collected data for five days from four participants (2 males and 2 females). Three of the four participants are residents, and the other is a carer. The data acquired from the carer were used only as training data. As shown in Figure 9, the participants wore our developed BLE beacon on their waist, as this did not obstruct their rehabilitation or recreation activities. The participants then went about their normal day. The data of each participant were collected by the flow described in Section 3.3. Our experiment has the following three purposes: (1) determination of effective machine-learning algorithm, (2) determination of effective smoothing method, and (3) selection of appropriate time window.
4.2. Environment
Ikoi-no-ie 26 is an actual nursing home that has a 250 m2 floor. Figure 10 shows its interior view, and Figure 11 shows its floor plan. In the experiment, we divided the floor into these six target areas: toilet, bath, rehabilitation, resting, recreation, and living. We determined them based on discussion with Ikoi-no-ie 26’s owner. The toilet and bath areas are closed spaces behind sliding doors, and other areas are open spaces. In the experiment, we tracked the four participants by installing nine scanners at the positions indicated by red circles (Figure 11). A server collected data from scanners and automatically generated activity reports for each participant. Carers can read the daily reports on their tablet or PC. We also installed two cameras to acquire groundtruth data (Figure 11).
4.3. Evaluation Method
We conducted the following four kinds of evaluations and explain their detailed results in Section 5. (a)Comparison of Machine-Learning Algorithms for Classification. We evaluated the classification performance of four kinds of machine-learning algorithms to determine the most effective one. We evaluated the classification performance of the machine-learning algorithms by comparing the recognition accuracy of each activity using leave-one-person-out cross-validation. The important factors in determining an effective machine-learning algorithm for classification are not only high activity recognition accuracy but also whether the toilet and bath activities (whose recognition we prioritized in the daily reports) are accurately recognized. Thus, we did not determine an effective machine-learning algorithm for classification by only average recognition accuracy; instead, we comprehensively determined it by investigating the recognition accuracy of each activity.(b)Comparison of Smoothing Methods. We evaluated the performance of two kinds of smoothing methods and determined the most effective one by adapting the smoothing methods (four patterns) to the machine-learning algorithm for the classification determined in evaluation (a). After discussions with Ikoi-no-ie 26’s owner, we determined the priority of each activity with machine learning for smoothing as follows: (1) toilet, (2) bath, (3) rehabilitation, (4) resting, (5) recreation, and (6) living.(c)Comparison of Time Windows. We investigated the effect of applying different time windows to the combination pattern determined by evaluations (a) and (b) to select an appropriate time window and determined a final combination pattern from the result.(d)Evaluation of Generated Daily Reports. Finally, we evaluated the effectiveness of the proposed system by comparing the daily reports generated by the groundtruth data and the final combination pattern determined by evaluations (a), (b), and (c). In this evaluation, we showed that the proposed system can incorporate new residents without data collection.
5. Result
5.1. Comparison Result of Machine Learning Algorithms for Classification
First, we evaluated the performance of the machine-learning algorithms for classification by comparing the recognition accuracy of each activity using leave-one-person-out cross-validation. Figure 12 compares the results of the machine-learning algorithms for classification. The horizontal axis shows each activity and the weighted average, and the vertical axis shows the -measure.
As shown in Figure 12, for the weighted average, all algorithms show a relatively high -measure over 0.65. SVM, RF, and GBDT keep relatively stable -measures for all activities. By contrast, LR shows an overall lower -measure than the above algorithms. In particular, the -measure for recreation and living activities, which have low priority, is relatively high, but the -measure for toilet and bath activities, which have high priority, is very low. In addition, the -measure for rehabilitation and resting activities, which have medium priority, is also very low. From these reasons, we excluded LR with low -measure in this evaluation. Hence, we determined SVM, RF, and GBDT as effective algorithms for classification in this study.
Figure 13 shows the daily reports (classification results) generated by the determined algorithms (before adapting the smoothing methods). Many misclassifications remain in each daily report that was generated by the selected algorithms. Therefore, we have to reduce them. We believe that daily reports with higher accuracy can be generated by reducing these misclassifications. Thus, we clarified which smoothing methods are effective by applying two kinds of smoothing methods to the determined algorithms in the next subsection.
5.2. Comparison Result of Smoothing Methods
We determined the most effective smoothing method by adapting the smoothing methods (four patterns) to the machine-learning algorithms for the classification determined in Section 5.1. Table 1 shows the comparison result of the smoothing methods (time ). The top row shows the machine-learning algorithms for the classification determined in Section 5.1, and the second row shows the machine-learning algorithms for smoothing. Note that CR shows the classification result before adapting the smoothing methods. Also, the numbers in Table 1 show the -measure. In this evaluation, we used a value of time seconds for all the smoothing methods to confirm the effect of each method.
After adapting the smoothing methods, we found that the -measure became quite high for all the combination patterns except CR. Therefore, applying smoothing methods is effective. However, when applying machine learning to smoothing, unrecognizable activities occurred. Smoothing by LR cannot recognize (i.e., ) the toilet, rehabilitation, and resting activities, regardless of which machine-learning algorithms we had used for the classification determined. Smoothing by RF or GBDT cannot recognize resting activities. In the daily reports that require detailed information, unrecognized activities obviously cause problems. Thus, machine learning for smoothing is not very effective.
On the other hand, smoothing by the most frequent value (MFV) tends to have a higher -measure for activities other than resting compared with CR, regardless of the machine-learning algorithms for the classification determined in Section 5.1. The -measure on resting activities tends to be lower than CR, but this does not mean that this activity cannot be recognized. Thus, since smoothing by MFV is more effective than smoothing by machine learning, we focus on it.
We compared the combination patterns of RF-MFV, GBDT-MFV, and SVM-MFV. SVM-MFV has a lower -measure in toilet and bath activities, which have higher priority than other combination patterns. In addition, this combination has a lower -measure for the weighted average than the other combinations. By contrast, RF-MFV and GBDT-MFV have higher -measures on toilet and bath activities than SVM-MFV. In daily reports, accurately recognizing information on toilet and bath activities is critical. Hence, in this evaluation, we determined that the combination patterns of RF-MFV and GBDT-MFV have high recognition accuracy of those activities and are the most effective combination patterns among 12 combination patterns.
5.3. Comparison Result of Time Windows
Next, we investigated the effect of applying different time windows to combination patterns determined by Section 5.2 to select an appropriate time window. Figure 14 shows a comparison result of time windows with this combination pattern: RF-MFV, and Figure 15 shows a comparison result of time windows with this combination pattern: GBDT-MFV. The horizontal axis shows the time window, and the vertical axis shows the -measure. In this evaluation, we investigated the effect of using different time windows of 0 to 120 seconds (13 patterns).
As shown in Figures 14 and 15, we found no significant improvement in the -measure of each activity even if any time window was used. In such a case, we need to select an appropriate time window based on priority. The most important factor when determining the appropriate time window is that the -measure is high on the toilet and bath activities that have high priority. Based on such a policy, from Figures 14 and 15, we decided the time window of each combination pattern as follows. The time window is 60 seconds for RF-MFV and 50 seconds for the GBDT-MFV. In this evaluation, we determined these patterns as final combination patterns among 156 combination patterns.
5.4. Evaluation Result of Generated Daily Report
We evaluated the effectiveness of our proposed system by comparing the daily reports generated by the groundtruth data and by the final combination patterns determined by Section 5.3. Figure 16 shows the daily reports generated by the final combination patterns. The upper graph shows a daily report generated by RF-MFV (time ). The middle graph shows a daily report generated by GBDT-MFV (time ). The lower graph shows a daily report generated by the groundtruth data.
From Figure 16, we confirmed that the proposed system can accurately generate daily reports to some extent using RF-MFV (time ) and GBDT-MFV (time ). RF-MFV (time ) can basically output toilet, living, and recreation activities. We also found that GBDT-MFV (time ) can exactly output toilet, bath, and recreation activities to some extent. However, we still see far too many misclassifications from the result in Figure 16. Such misclassifications are not serious because we can relatively accurately grasp a participant’s activity for a single day. Hence, we conclude that the final combination patterns are sufficiently effective based on these results.
Finally, Table 2 shows the evaluation result for the three participants who are Ikoi-no-ie 26 residents. For the weighted average, the proposed system has high scores that exceed approximately 0.80 in each precision, recall, and -measure index. Participants A and C have especially high scores for each index. Also, participant B has the high score for each index, although it is somewhat lower than the scores of participants A and C. We achieved these results by evaluating the proposed system with leave-one-person-out cross-validation. In other words, this result means that the proposed system can incorporate new residents without data collection. Hence, it satisfies requirement 5 that we discussed in Section 2.3. This result also shows that the proposed system can accurately track the activity of multiparticipants. It also satisfies requirement 1 in Section 2.3. Hence, our proposed system sufficiently achieved the research goals of this study.
6. Discussion
In the evaluations in Section 5, we determined RF-MFV (time ) and GBDT-MFV (time ) as the most effective among 156 combination patterns. We found that the daily reports generated by these effective combination patterns are accurately output to some extent. The same result was obtained for other residents. These results mean that the proposed system simultaneously monitored the activity of multiresidents and automatically generated daily reports corresponding to each resident. Thus, our proposed system sufficiently achieved our research goals. Also, this system sufficiently satisfied all the requirements described in Section 2.3 from the system design and evaluation results. In this section, we discuss the usability and scalability of our proposed system to derive a clearer conclusion in this paper.
6.1. Usability
First, we discuss the usability of the proposed system. To investigate its usability, we discussed it with the owner of Ikoi-no-ie 26 and interviewed the experiment’s participants about the mobility of our BLE beacon. We obtained the following results from the owner and participants: (i)The proposed system was an adequate prototype for Ikoi-no-ie 26, since the activities of each resident and new resident were visualized without any manual work by carers. It satisfies requirements 1 and 5 in Section 2.3.(ii)The developed BLE beacon used in the proposed system was light enough for seniors to carry. It satisfies requirement 2 in Section 2.3.(iii)The proposed system raises no privacy concerns since it does not use cameras or microphones. It satisfies requirement 3 in Section 2.3.(iv)The proposed system is easy to introduce even in a small nursing home, since it is compact and inexpensive. It satisfies requirements 4 in Section 2.3.
The above results satisfy all requirements in Section 2.3 and demonstrate that our proposed system is sufficiently helpful for the nursing duties/tasks of carers in nursing homes. Furthermore, it places a small burden on users and low privacy concerns for residents. Hence, since it is very easy-to-use for both carers and residents, it has high usability.
6.2. Flexibility
Next, we discuss the flexibility of our proposed system. Especially, we believe the flexibility of our developed BLE beacon is very high. Since our BLE beacon has an accelerometer with a 3-axis accelerometer, a gyroscope, and a geomagnetic sensor, we can obtain various useful information, for example, detailed activities and the number of steps by analyzing the obtained data of each sensor. We believe that utilizing our BLE beacon will be used for rehabilitation support in the future in aging societies. Here, we list three examples that show the flexibility of our BLE beacons: (i)The developed BLE beacon has a classifier that recognizes the following activities: “sit,” “stand,” and “walk.” It can recognize the kind of activities performed by nursing home residents. More detailed daily reports might be generated by utilizing the results of these activity recognitions [2, 9, 10].(ii)The developed BLE beacon can measure the number of steps of each resident by analyzing the obtained acceleration data. Perhaps the physical activity of each resident can be grasped by measuring these steps. The physical activity recognition of residents is also important for the daily reports, since the ratio of a day’s activity is the key information to obtain the states of their health.(iii)The developed BLE beacon has a 3-axis accelerometer, a gyroscope, and a geomagnetic sensor. Therefore, perhaps the type of rehabilitation can be recognized by integrating and analyzing these sensor data. If our BLE beacon can recognize the type of rehabilitation, the completeness of the daily reports will be further improved.
Hence, we conclude that our proposed system has high flexibility.
7. Literature Review
There are many studies related to our proposed system. In this section, we discuss them in the following three subsections: (1) location estimation technique, (2) activity recognition technique, and (3) activity report generation in health care.
7.1. Location Estimation Technique
Location estimation techniques have been proposed based on several kinds of techniques, including Wi-Fi signals [26], BLE signals [3, 4, 27], magnetic fields [28], infrared [29], and ultrasounds [30]. For example, camera-based systems are one of the most famous techniques [31, 32]. However, they are not suitable for our target environment because they cannot distinguish people unless their face is captured clearly. If we set several cameras in the corners of the center room, the person’s face captured by the camera is too small to be recognized. If we set so many cameras, costs are increased, and personal privacy is violated as camera accuracy increases. Thus, they are unsuitable for our target environment.
Location estimation techniques based on passive RFID systems have also been proposed [33–37]. These techniques mainly estimate location using a probabilistic model called a sensor model or the directivity formed by the combination of a reader antenna and an RFID tag. However, since they require at least a reader antenna or a special device to receive radio waves from passive RFID tags, we cannot use them in our target environment because they increase the burden on residents. In addition, these studies did not simultaneously estimate the locations of multiple people or objects.
Location estimation techniques based on Wi-Fi and/or BLE signals have also been proposed [38–42]. These techniques achieved comparatively high localization accuracy by estimating location based on triangulation, fingerprints, signal strength (SS), time difference of arrival (TDOA), time of arrival (TOA), and angle of arrival (AOA). However, they require such devices as smartphones to receive Wi-Fi and/or BLE signals. Therefore, the burden on residents is increased. So, we cannot use them in our target environment.
7.2. Activity Recognition Technique
Some studies focus on activity recognition [17, 43, 44]. Activity recognition techniques that use wearable accelerometers have already achieved accuracies exceeding 90% for such simple activities as walking, sitting, running, and sleeping [45]. Lee and Mase [46] proposed a technique to estimate “walk,” “run,” and “sit” with an accelerometer and a gyroscope attached to users. The technique of Bao and Intille [47] used five wearable accelerometers and recognized 20 activities, such as watching TV, cleaning, and working. Ueda et al. [48] introduced an activity recognition system in a smart home utilizing an ultrasonic sensor-based indoor positioning system and power meters attached to each home appliance.
To utilize the above techniques, residents have to carry a smartphone or several wearable devices. However, they dislike carrying them, as we described in “location estimation technique.” Moreover, most collect acceleration data with smartphones and conduct machine learning using high-performance computers. Thus, these techniques are not suitable for our target environment.
7.3. Activity Report Generation in Health Care
Other studies addressed activity report generation in health care [49–51]. Inoue et al. [12] investigated the activity recognition of nurses in hospitals and estimated their activities with accelerometers by mobile devices carried by nurses and Bayesian estimation. They report that one of the most time-consuming duties is the documentation required for nursing reports. Documentation in both hospitals and nursing homes is a bottleneck that must be removed before service quality can be improved. However, these studies assume that users always carry mobile devices such as smartphones. Furthermore, since these studies cannot respond to new residents, they are unsuitable for our target environment.
8. Conclusion
We proposed an automatic daily report generation system that monitors the activity of residents in nursing homes. The proposed system estimates the current area of multiple residents utilizing machine learning with the RSSI values of BLE signals as a feature quantity and recognizes the activity of each resident from the estimated area information. The information of the recognized activity of each resident is stored in the server with timestamps. The system automatically generates daily reports corresponding to each resident based on them. To show the effectiveness of our proposed system, we conducted an experiment at an actual nursing home. The following are our primary findings: (i)The RF-MFV (time ) and GBDT-MFV (time ) combination patterns are the most effective among 156 combination patterns.(ii)The proposed system can accurately generate daily reports that correspond to each resident to some extent using the above combination patterns.(iii)The proposed system can accurately track the activities of multiple participants.(iv)The proposed system can incorporate new residents without data collection.(v)The proposed system has high usability and high flexibility based on the discussion in Section 6.
Since our proposed system satisfactorily achieved our research goals, we conclude that it can sufficiently assist the daily work of carers in nursing homes. On the other hand, the problem of obtaining information (especially food and information about vital signs) that we cannot obtain in our proposed system is critical. Future work will develop a system that easily obtains such information and encourages input at precise timing to terminals possessed by carers when residents and carers enter an area where the above information must be input.
Data Availability
The data used to support the findings of this study have not been made available because the authors have a contract for the data usage with the nursing home that was used as the experimental facility.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work is partly supported by the Japanese Government Monbukagakusho: Japan Society for the Promotion of Science KAKENHI Grant nos. 16H01721 and 16K00126. Also, the authors would like to thank Mister Yukawa, the owner of Ikoi-no-ie 26, for giving them many attractive advises to promote this work as well as providing Ikoi-no-ie 26 as a testbed for this research.