To reduce the risk of elderly people falling in a private room without relying on a closed-circuit television system that results in serious privacy and trust concerns, a fall monitoring system that protects the privacy and does not monitor a person’s activities is needed. An ultrasonic-based sensor system for elderly fall monitoring in a smart room is proposed in this study. An array of ultrasonic sensors, whose ranges are designed to cover the room space, are initially installed on a wall of the room, and the sensors are rotated to transmit and receive ultrasonic signals to measure the distances to a moving object while preventing ultrasonic signal interference. Distance changes measured by ultrasonic sensors are used as time-independent patterns to recognize when an elderly person falls. To evaluate the performance of the proposed system, a sensor system prototype using long short-term memory was constructed, and experiments with 25 participants were performed. An accuracy of approximately 98% was achieved in this experiment using the proposed method, which was a slight improvement over that of the conventional method.

1. Introduction

The world population, which was approximately 7.7 billion people in mid-2019, is gradually increased [1]. The number of elderly people will dramatically rise to approximately 11.1%–18.6% of the total population in the next 30 years [2] due to successful birth control policies and advanced medical care services. As a result, the ratio of the working population to the elderly population is rapidly decreasing. This will lead to a worldwide labor shortage for many industries, including elderly care. At the beginning of the twenty-first century, the European commissioner launched a project called the TeleCARE consortium [3] to design and develop a virtual community infrastructure platform for elderly support. Many other countries in other regions also launched similar projects.

In the community infrastructure, the bedroom, living room, and restroom are the main rooms where elderly people spend most of their time, especially when they are at home, and it is assumed that during this time, privacy is preferred [4]. As elderly individuals are more susceptible to fatal fall [5], they need immediate assistance when a fall occurs. Therefore, automatic fall detection and alarm systems, which are used instead of caregivers to continuously monitor elderly individuals in rooms, must be developed under the conditions of privacy and trust.

The authors of this study proposed the development of an elderly fall monitoring system that can be installed in a private room. The basic conditions of this system include trust, privacy, and safety. We attempted to utilize ultrasonic sensor arrays installed on the ceiling and walls of a room to detect human falls, and experiments were performed with a static model [68]. A small room (30 × 30 cm2) with nine ultrasonic sensors was used to test the static model [6]. Ultrasonic interference between neighboring sensors and sensor calibration were found to be problems. These problems were considered, and the sensors were adjusted to mitigate interference [7]. However, coverage gaps between neighboring sensors in the array were found. The overlap of neighboring sensors was proposed to address these gaps, and experiments were performed using the model in a larger room (60 × 60 cm2) [8]. It was confirmed that the methods proposed in this study could be used to recognize falls using a static model. We also attempted to construct a prototype system for human fall detection. The duration of a fall is approximately 0.7 s. Accordingly, the system must be improved to be able to complete all required processes during this short period of time. Therefore, we reconsidered the system with respect to the hardware and software, improved the hardware, installed an ultrasonic sensor on a sidewall, and developed an algorithm to sense distances to humans in a point-by-point manner to detect falls using the state transition concept without identifying the specific behaviors of the person.

This study is organized as follows. The related works and smart room scenario, including the analysis of ultrasonic array sensors, are presented in Sections 2 and 3, respectively. Our proposed design method is described in Section 3. Experiments and results are reported in Section 4, and a discussion is presented in Section 5. Finally, the study is concluded in Section 6.

Based on the research problem of elderly fall detection, many researchers have attempted to develop algorithms and systems to monitor elderly individuals and detect their falls. As shown in Table 1, research works related to the development of fall detection systems are generally divided into three approaches [37, 38]: vision-based approaches [1420, 3941], wearable device approaches [2138], and ambience sensor approaches [68, 3941, 4269]. The vision-based approach is considered an excellent approach due to the beneficial 3D shape information it provides. Accurate results have also been achieved using the wearable device approach because of the dynamic personal position information obtained. Due to the sensors and their locations, sensed information using ambience sensor approaches is mainly a point-to-point approach, which is a limitation. However, the methods in this group were not originally utilized to clearly recognize an object shape. Therefore, these methods are considered to better protect resident privacy.

Previous fall detection systems using ambience devices, including infrared (IR) [3941], IR and ultrasonic [42, 43], radar [4451], Wi-Fi [5257], sound [5862], and ultrasonic [4955] systems, have been developed, and their advantages and disadvantages are discussed as follows. IR is mainly used to confirm the absence of objects [3941, 42, 43], but it cannot be used to measure distance, which makes 3D reconstruction difficult. Moreover, IR has been proven to be harmful to the human eyes and body [3941, 42, 43]. As it is widely used, Wi-Fi is convenient. Radar is an excellent technique for scanning moving objects for a long time. However, both Wi-Fi and radar operate at high frequencies, which harm the human body and especially affect heart attack patients who use pacemakers [63, 64].

Other than the hearable frequency range, which causes noise harm, sound does not cause any serious harm to the human body. Furthermore, the humans cannot hear an ultrasonic sound. Therefore, sound can be safely used to measure distance using its reflected wavelength [65]. As ultrasonic sound is comparatively safe for the human body, it is recommended in this study as a good technology to develop an automatic fall detection system. In fact, ultrasonic signals, which are mainly applied in point-to-point distance measurements by directional characters, currently cannot be developed at a high resolution for object recognition due to the constraints of its beam. This observation is conversely a merit for this research problem, since, in general, elderly people prefer privacy in their living environment. The ultrasonic sensor becomes a solution to the research problem.

In the research and development of the automatic fall detection systems related to ultrasonic sensors, as shown in Table 2, Toshio Hori et al. [66] presented ultrasonic sensors, which utilized the speed of falling in a vertical direction, for elderly people and their caregivers in a nursing home. This method worked, especially in vertical falling applications, but might not be suitable for some complicated cases of diagonal falling, which is a limitation. Yirui Huang et al. [67] suggested a method to detect falls and remote activity using ultrasonic sensors. This work focused on a cost-effective and intelligent hardware design for individual ultrasonic sensors. This method also utilized the speed and level of falling in each sensor so that it was not matched with complicated fall patterns. Chang and Shih [68] proposed human fall detection based on event pattern matching with ultrasonic array sensors. This method may be used to detect many human fall patterns. However, the sensors were installed at the height of the human leg, which may not be robust enough to detect all fall patterns. Ghosh et al. [69] proposed UltraSense, which is used to identify human activity using a heterogenous ultrasonic sensor grid for smart home environments. This system identified human activity well, which is not preferred from a privacy viewpoint. Moreover, the ultrasonic sensor grid was installed on the ceiling of the room. Since height information is used, the difference between a fall and some normal activities in the vertical direction may confuse the system.

3. Smart Room Scenario

According to statistical data from the United Nations (UN) in 2019 [70], all societies in the world are in the midst of a “longevity revolution,” and the number of elderly people worldwide has dramatically increased year by year. In addition, children and relatives who may closely care for elderly family members do not reside with them in the same household. Approximately half of adults aged 60 and older live alone and approximately one-third live with a spouse only (without children or relatives) in North America and Europe [71]. When an elderly person on their own has a vital accident, such as a fall, it is difficult to obtain assistance as other people may not have noticed that the accident occurred. Unless someone notices and helps the person in a timely manner, an unexpected death may occur.

To solve this problem, an automatic monitoring system that protects user privacy and does not monitor activities should be installed in every house. In addition, the system must provide an immediate alert whenever a fall occurs. In our proposed scenario, an automatic monitoring system, with sensors and devices that do not directly visually recognize and record activities, is installed on the wall of a room. Therefore, an ultrasonic node was selected as the sensor in this study to continuously measure distances from the ultrasonic nodes to the person in the room in a point-to-point manner. In addition, external memory, which can store data permanently, is not utilized in this system to protect activity data. As shown in Figure 1, the fall monitoring system consists of a processor, microcontroller, and sensors that are installed in a room, which is called a smart room in this study. The system may immediately alert caregivers, relatives, and children via Wi-Fi, the cloud, and mobile phones whenever a fall occurs.

Based on the scenario of a smart room for elderly people, privacy-based and distance-based human fall detection and sensor blind zone are analyzed as follows.

3.1. Privacy-Based Human Fall Detection

According to the studies in [72], the human body comprises moving-independent parts or modules, and a point located on a module called a control point can represent the module position, as shown on the right side of Figure 1. If a control point representing a module is detected and the distances from the sensors to the control points are measured, the body of a person is detected and monitored. If all control points of the person are continuously detected and the distances are measured all the time, then the person’s behavior is also monitored. Therefore, the patterns of control points when a fall occurs (e.g., a forward fall, a backward fall, a sideway fall, and a fall from a chair) can be observed and analyzed, as shown in Figure 2. These patterns can be trained in advance and can possibly be used to recognize falls in elderly fall monitoring systems. Although the control points of a person do not show real pictures such as a video clip captured by a closed-circuit television (CCTV), it may be possible to estimate the behaviors of the person even by the control point image. Elderly people may prefer that activities done during their private time are not monitored, as an absolute condition to develop the fall monitoring system.

Suppose that a video clip and an image of control points of humans are not allowed to be used as input data in the fall monitoring system. Instead, a point representing human body movement per frame, which contains the least amount of data in a frame and from which, it is almost impossible to determine activities, should be considered to enable fall recognition. During a fall, a point that is closest to a sensor is sensed, and this sensor may change based on the movement of the falling person. Distances from those points closest to a sensor that is continuously collecting information during a fall can be categorized into patterns based on the type of fall. Figures 3(a)–3(d) show some examples of video frames (upper row) and distances between sensors and a person (represented as a graph). The distance changes during a fall based on different fall types, e.g., forward, backward, and sideway falls and a fall from a chair, can be differentiated as patterns in the graph. These different patterns can be used to classify a fall and a nonfall as well as recognize fall types. If a classifier is trained with these distance change patterns in advance, pattern matching between these trained patterns and input data can always be used to classify the input as either a fall or a nonfall.

3.2. Distance-Based Human Fall Detection

To apply the abovementioned concept to the human fall problem, since the time period for a human fall is as short as approximately 0.7 s [72], the whole room space should be sensed and monitored so that the fall detection system can be processed and an alarm immediately activated during this period. In the case that a distance sensor is installed on a sidewall to measure the shortest distance from a sensor to a human at a point (s) in the room space, as shown in Figure 4, the change in measured distances during the fall duration () is mathematically expressed as the following fall pattern (F).

To cover the whole room space with ultrasonic signals without interference, multiple sensors are installed in an orderly manner in a matrix form, as shown in Figure 5. In this case, coverage ranges almost cover the sensing wall, but blind spaces or gaps exist between neighboring cells. To fill in the blind spaces, ultrasonic nodes should be shifted in an orderly manner along a straight line and simultaneously the given distance between consecutive ultrasonic nodes should be maintained to prevent interference, as shown in Figure 6(a). Additionally, the sensors can also be shifted in zigzag scanning lines to maintain balance in the horizontal and vertical directions. As an example, an ultrasonic node or more than one ultrasonic node can be scanned along a zigzag line, as illustrated by the black and red dashed arrows in Figure 6(b).

Therefore, a human fall pattern can be expressed by distance changes during a fall:where and are the ultrasonic node positions in the horizontal and vertical directions, respectively, and is the human fall duration.

This fall pattern can then be converted into a digital form to realize the results using a digital computer.

3.3. Geometric Concept for Sensor Array Installation

From the basic condition of the ultrasonic signal application, which must be generated without interference, the coverage range (k) on the sensing wall, as shown in Figure 7, can be obtained:where l and θ are the room width and sensor transmission angle, respectively.

The coverage range can be used to determine the number of ultrasonic generating nodes in the first static step (t1). Since it is assumed that nodes were shifted to cover all blind zones on the wall as previously described, the distance between a pair of ultrasonic sensor nodes will be calculated based on the condition of an unavoidable blind zone (B), as shown in Figure 8.

To apply this concept in fall detection applications, users should consider setting blind zones between consecutive ultrasonic rays at a distance smaller than the width of the human as the design condition. This guarantees the detection of human falls even in a blind zone. Thus, the range (d) between consecutive ultrasonic nodes can be obtained.

In addition, the power transmitted by a string wave (E) can be determined as follows [73, 74]:where μ, ω, A, and represent the mass per unit length of the string, angular frequency of the wave, wave amplitude, and wave propagation velocity, respectively.

4. Proposed Ultrasonic-Based Human Fall Monitoring System

Based on the abovementioned concept, a fall detection design and implementation method using ultrasonic sensors for the monitoring system is explained in this section. It is assumed that the system must be not only nonintrusive, noninvasive, and device-free but also protect the user’s privacy. In the following, the system design and implementation are divided into hardware and software parts.

4.1. Hardware Design

The hardware system mainly consists of two units, a sensor array and a signal processing unit. The sensor array installed on a wall must be designed to cover the whole room with the smallest blind zone. In addition, the signal processing unit must provide enough ports for receiving signals from all sensors and must be designed to have enough ability to process those signals. The design of the sensor array and signal processing unit is explained as follows.

4.1.1. Sensor Array

Suppose the scale of a room in which the ultrasonic sensor array for the human fall monitoring system is installed is M × N × l. Ultrasonic sensor nodes should be geometrically installed on a sensor wall (M × N) in the room under the condition of noninterference. The range between consecutive ultrasonic nodes in the horizontal and vertical directions can be simply determined.

If B represents the blind zone for human sensing, which has to be determined in advance, the range between consecutive sensors in the array (d) can be obtained based on (5).

In the first step, the number of sensors in the horizontal and vertical directions for sensing the distance in a frame, which must be limited due to the interference of ultrasonic signals from different sensors in a frame, can be simply determined.

Based on our proposed concept to shift the active sensors in the dense ultrasonic sensor array instead of dynamically scanning the ultrasonic sensors, the time duration for shifting mainly depends on the average human fall duration . If a sensing process takes a duration of time , a shifting range (d), which is regarded as the dense sensor node range, can be calculated.

The total number of dense node sensors is therefore determined.

4.1.2. Signal Processing Unit

The signal processing unit consists of a power supply unit, an ultrasonic sensor array, a microprocessor, memory, and a classifier, as shown in Figure 8. The ultrasonic sensor array, which is supplied power by the power supply unit, always senses a moving object in the room. Analog signals representing the distance from ultrasonic sensors to a moving object are transmitted to the microprocessor. In the microprocessor, the analog signals are translated into distance data and stored in terms of a matrix in the memory. The matrix of distance data is finally fed to the classifier as a feature for fall classification.

In the power supply unit, the power needed to drive an ultrasonic sensor can be calculated based on equation (6). In equation (6), , and can be determined as the mass per unit length of the string [75], the frequency of the ultrasonic signal (defined above the sound frequency or set to 20 kHz), the amplitude of the ultrasonic signal, and the ultrasonic velocity [76], respectively.

Since this study proposes the utilization of the change patterns of distance from the ultrasonic sensor nodes to the closest point of a person who has fallen, the ultrasonic signals should be transmitted to reflect the falling person in as many round trips as possible. The number of scanning frames () for ultrasonic signal transmission in one second can be simply estimated:where are the ultrasonic signal velocity, frequency, processing time, and room depth, respectively.

In human fall detection and classification, the number of scanning frames is one of the crucial keys to fundamentally guaranteeing quality. The number of scanning frames is practically an initial condition to design the signal processing unit. Although the more scanning frames there are, the more robust the system is, and users may select an appropriate set of devices that is normally limited by processing time. For example, the processing time of a frame is limited on an approach of real time. Thus, the number of scanning frames should be calculated in real time during the period of a human falling, which is approximately 0.7 s. Suppose users fix the number of scanning frames, the maximum processing time allowed in each frame is what users may need next to select electronic devices in the hardware design step. Thus, users can expect to be able to select devices by specifications based on the allowed processing time. The processing time per frame (P) of devices allowed in the system design can be calculated.where are the ultrasonic signal velocity, frequency, number of scanning frames, and room depth, respectively.

4.2. Software Design

The software system for retrieving ultrasonic signals representing the distance to a moving object can be designed and created, as shown in Algorithm 1. First, the initialization and declaration of variables are registered for values measured by the sensors and shown in steps 2–4. Then, an infinite loop (steps 5–14) is run to read a distance value on an ultrasonic sensor, store it in a matrix, shift the active sensor to another neighboring sensor according to the zigzag direction in Step 6 for all sensor nodes, and then classify the fall. The 2D matrix of distances in Step 6 is converted into a 1D matrix in steps 7–12, and it is fed to a classifier for fall classification in Step 13. The details of the rotation of active ultrasonic sensors and fall classification are explained in Sections 4.2.1 and 4.2.2, respectively.

Input: m: number of nodes in row, n: number of nodes in column,
    for col = 1 to n do
    End for
  End for
  COMPUTE: Classification based on input data of measured distances
UNTIL all signal samples
4.2.1. Rotation of Active Ultrasonic Sensors

To implement a software unit of the fall monitoring system, multiple ultrasonic nodes in an array are expected to simultaneously sense distances from the sensors to objects. However, interferences among ultrasonic signals may occur and lead to errors in the case where coverage areas overlap. To prevent ultrasonic signal interference, coverage areas must not overlap. The possible number of ultrasonic nodes for the simultaneous distance measurement can be determined by the minimum distance between working ultrasonic nodes, as shown in equation (7). To efficiently scan ultrasonic nodes in an array by maintaining a minimum distance, zigzag scanning is recommended to balance the horizontal and vertical directions; notably, zigzag scanning is demonstrated for a pair of ultrasonic nodes in Figure 8. Suppose that a couple of scanning lines on ultrasonic nodes in an array simultaneously start from points A and B; they synchronously move along a zigzag scanning line to the next nodes, which are labeled in the same colors. If the coordinates of ultrasonic sensor nodes are represented by row (R), column (C), and current node counting (i), patterns of nodes moving in a couple of zigzag scanning lines (A and B) can be logically illustrated by rows (RA, RB) and columns (CA, CB) of A and B.

Group 1. Shift from the first node (represented by a scanning line A in Figure 9)
Initially, the scanning node will be shifted to the adjacent node in the edge row as initial couple nodes. This logic can be simply expressed as follows.Pattern a:IF I = 0 THEN RA(i+1) = RA(i), CA(i+1) = CA(i) + 1,//A : Shift straight right. RB(i+1) = RB(i) + 1, CB(i+1) = CB(i) 1.//B : Shift diagonally left down

Group 2. Cases where the current sensing node is located in the 0th row (RA(i) = 0); Subgroup 2.1 Shift in the diagonal down direction (b on scanning line A in Figure 9)
When the previous sensing node (RA(i–1), CA(i–1)) located in the 0th row is shifted right to the current sensing node (RA(i), CA(i)) located in the 0th column, the next sensing node (R(i+1), C(i+1)) will be shifted down in the diagonal direction. This logic can be expressed as follows.Pattern b:IF {RA(i) = 0} and {|RA(i)RA(i–1)| = even} and {|CA(i)CA(i–1)| = odd}THEN RA(i+1) = RA(i)+ 1, CA(i+1) = CA(i) – 1,//Shift diagonally left downRB(i+1) = RB(i), CB(i+1) = CB(i) +  1.//Shift rightSubgroup 2.2. Shift right on the 0th row (f on scanning line A in Figure 9)
When the previous sensing node (RA(i–1), CA(i–1)) located outside of the 0th row approaches the current sensing node (RA(i), CA(i)), the next sensing node (RA(i+1), CA(i+1)) will be shifted right on the 0th row. This logic can be expressed as follows.Pattern c:IF {RA(i) = 0} and {|RA(i)RA(i–1)| = even} and {|CA(i)CA(i–1)| = odd}THEN RA(i+1) = RA(i), CA(i+1) = CA(i)+ 1,//Shift rightRB(i+ 1) = RB(i)+ 1, CB(i+1) = CB(i) – 1.//Shift diagonally left down

Group 3. Cases where the current sensing node is located in the 0th column (CA(i) = 0); Subgroup 3.1 Shift down on the 0th column (c on scanning line A in Figure 9)
When the previous sensing node (R(i–1), C(i–1)) located out of the 0th column approaches the current sensing node (R(i), C(i)) located on the 0th column, the next sensing node (R(i+1), C(i+1)) will be shifted down on the 0th column. This logic can be expressed as follows.Pattern d:IF {CA(i) = 0} and {|RA(i) – RA(i–1)| = odd} and {|CA(i) – CA(i–1)| = even}THEN RA(i+1) = RA(i+1), CA(i+1) = CA(i),//Shift downRB(i+1) = RB(i) 1, CB(i+1) = CB(i) +  1.//Shift diagonallyShift in diagonal right up direction (d on scanning line A in Figure 9)
When the previous sensing node (R(i–1), C(i–1)) located out of the 0th column is shifted down to the current sensing node (R(i), C(i)) located on the 0th column, the next sensing node (R(i+1), C(i+1)) will be shifted up in the diagonal direction. This logic can be expressed as follows.Pattern e:IF {CA(i) = 0} and {|RA(i) – RA(i–1)| = odd} and {|CA(i) – CA(i–1)| = even}THEN RA(i+1) = RA(i) 1, CA(i+1) = CA(i) +  1,//Shift diagonallyRB(i+1) = RB(i) – 1, CB(i+1) = CB(i) +  1.//Shift diagonally right

Group 4. Cases where the current sensing node is not located on the edge (CA(i)) ≠ 0 and (RA(i)) ≠ 0; Subgroup 4.1 Shift in the diagonal right direction (e on scanning line A in Figure 9)
When the previous sensing node (RA(i–1)), CA(i–1)) is shifted diagonal right up to the current sensing node (RA(i), CA(i)) located out of the 0th column and out of the 0th row, the next sensing node (RA(i+1), CA(i+1)) will be shifted up in the diagonal direction. The logic can be expressed as follows.Pattern f:IF {CA(i) ≠ 0} and {RA(i) ≠ 0} and {|RA(i) – RA(i–1)| = odd}and{|CA(i) – CA(i–1)| = odd}THEN RA(i+1) = RA(i) – 1, CA(i+1) = CA(i) +  1,//Shift diagonally up. RB(i+1) = RB(i) +  1, CB(i+1) = CB(i).//Shift downSubgroup 4.2. Shift in the diagonal left direction
When the previous sensing node (RA(i–1), CA(i–1)) is shifted diagonal right up to the current sensing node (RA(i), CA(i)) located out of the 0th column and out of the 0th row, the next sensing node (RA(i+1), CA(i+1)) will be shifted down in the diagonal direction. This logic can be expressed as follows.Pattern g:IF {CA(i) ≠ 0} and {RA(i) ≠ 0} and {|RA(i) – RA(i–1)| = odd} and {|CA(i) – CA(i–1)| = odd}THEN RA(i+1) = RA(i) +  1, CA(i+1) = CA(i) – 1,//Shift diagonally left down. RB(i+1) = RB(i) – 1, CB(i+1) = CB(i) +  1.//Shift diagonally right upThese logics cover the possible movement of a couple of current sensing nodes (RA(i), CA(i)) and (RB(i), CB(i)) to the next sensing nodes (RA(i+1), CA(i+1)) and (RB(i+1), CB(i+ 1)) in a couple of zigzag scanning lines. As an example, an algorithm for shifting a couple of sensing nodes to the next sensing nodes in a couple of zigzag scanning lines is described in Algorithm 2.

4.2.2. Fall Classification

Due to the time limitation and time invariance of a human fall, long short-term memory (LSTM), which is regarded as an excellent classifier for time invariance, is utilized in this study. Distance information from sensors to a human is assumed to be input data for the classifier. As shown in Figure 10, the node number in the input layer is determined based on the number of sensors that sense a moving object during a human fall, which is assumed to take approximately 0.7 s, and the output nodes for the fall detection system should be set as many to one, with a number of preference choices, such as a backward fall, forward fall, or walking. The hidden layer of the LSTM is used to set the input size per series to 2/3 of its original value [77].

READ: distances from A and B
SHIFT A straight right
SHIFT B diagonally left down
READ: DISTANCE distances from A and B
  Switch Pattern do
   case b
     SHIFT A diagonally left down
     SHIFT B straight right
   case c
     SHIFT A straight right
     SHIFT B diagonally left down
   case d
     SHIFT A straight down
     SHIFT B diagonally right up
   case e
     SHIFT A diagonally right up
     SHIFT B diagonally right up
   case f
     SHIFT A diagonally right up
     SHIFT B straight down
   case g
     SHIFT A diagonally left down
     SHIFT b diagonally right up
  READ: distance
END for

Pretests must be performed on some samples using possible parameters in the training state to determine the experimental parameter settings. As shown in Table 3, the activation function (e.g., softmax, ReLU, sigmoid, and tanh) should be pretested on some samples in advance and selected appropriately for the networks. In the input layer, the batch size, input size per series, input feature, and learning rate are trained with a number of within the capability of the graphics processing unit (GPU) memory, total number of input data of all features, input data dimension, and appropriate rate for gradient descent that considers an appropriate time, respectively, without overshooting [78].

5. Experiments and Results

To evaluate the performance of the proposed method, a room with ultrasonic sensors installed was constructed, and a data processing system was implemented based on the experimental specifications, as shown in Table 4. For the experiments, a representative group of participants were selected based on sex and age. These participants were trained to walk, sit, and fall in the room before experiments were performed. The LSTM classifier was set up based on the specifications shown in Tables 1 and 2. Photographs of an empty room, a room with a participant, and our design interface with a memory card are shown in Figures 11(a)–11(c), respectively. The experiments were performed by 25 participants using some behavior criteria, including falling, walking, and sitting in the constructed room, as shown in Table 5. The experimental results obtained in distance data of one and two points per frame reveal the human fall recognition rate based on the number of training and testing samples, as shown in Tables 6 and 7, respectively. The results are divided into the following groups: 20–40 years of age and 41 years of age and older. Training and testing data from the experiment were processed with ratios of 90 : 10, 80 : 20, 70 : 30, 60 : 40, and 50 : 50 to access the accuracy of the training and testing criteria. An accuracy of approximately 99.14% was achieved using the ratio of 90 : 10 after the training and testing experiments. Compared with conventional methods, the proposed method exhibited an improvement in accuracy by approximately 1.14%, as shown in Table 8. Examples of distances captured by two node sensors in the cases of a forward fall, backward fall, fall from a chair, and walking during 1,000 ms are shown in Figure 12. These graphs show patterns of distance changes for each case. While ultrasonic signal transmitters are located on a wall scan in the zigzag direction to transmit ultrasonic signals by two nodes each time, distances are always measured by 16 sensors (0,0–3,3) located on the opposite sidewall. All measured distances during 1,000 ms are shown in the graphs. In these measured distance data, differences among a forward fall, backward fall, sideway fall, fall from a chair, and walking were observed. These data were input into the LSTM for training and classification.

In the experiments, the measured distances in continuing frames that showed behaviors, such as falling, walking, and sitting done by the 25 participants (as shown in Table 4), were used to train and test with various ratios. Errors occurred based on the use of one node or two nodes, as shown in Table 9, respectively (Table 10). These errors are analyzed in the discussion.

6. Discussion

To build a smart room that can be used to detect the falls of elderly people without an intrusion of privacy, this study proposes installing an array of ultrasonic sensors on a wall, activating the sensors to sense distance information, and classifying the falls of elderly people based on distance change patterns. The performance of the proposed method was evaluated, and the accuracy was more than 90% in the cases of training more than 50% of the 2 node-based sample data, as shown in Table 7. Based on the results of the 1 node-based data shown in Table 6, the accuracy using this method was worse than that of the 2 node-based data because sensing based on one node was insufficient to cover the whole room. If the room was much smaller so that the range of the ultrasonic signal covered the room, one node would be adequate. Users must consider the coverage range of the ultrasonic signals with respect to the room scale as one of the design conditions. The results shown in Tables 5 and 6 confirm the effective range of sensor coverage and were used to evaluate the performance of the proposed method. Acceptable accuracy was achieved. The accuracy increased according to the increasing ratio of training samples, and an approximately 98% accuracy was achieved using the 90 : 10 training and testing ratio. Although elderly individuals (individuals over 40 years of age in this experiment) comprehensively caused both positive and negative faults, they contributed a large number of true positives to the results compared with the other group. Therefore, the proposed method was considered to be applicable to elderly individuals. The proposed method was compared with conventional methods, and it was obvious that higher precision and recall were obtained using the proposed method, as shown in Table 8. These high accuracy results were analytically caused by the distance change patterns of forward, backward, and sideway falls, falls from a chair, and especially walking, as shown by the examples in Figure 12. As observed change patterns among consecutive frames in the time domain, classifiers for video were confirmed to be appropriate tools for this kind of fall detection and classification problem. Analytically, errors decreased when the number of training samples was increased compared with testing samples, and accuracy was considered reliable in the 90 : 10 training and testing ratio, as shown in Tables 9 and 10. Therefore, the proposed method was proven to be effective for fall classification.

In addition, the fault-positive (FP) and fault-negative (FN) errors shown in the middle column of Tables 6 and 7 were analyzed, and causes of these errors were found, as shown in Table 11. The FP column in Table 10 indicates the number of positive errors in many error patterns, such as misclassifying sitting as falling and misclassifying the type of fall. These errors may cause a caregiver to be alerted to provide help to an elderly person who has fallen. Although these errors were considered a waste of time and energy for the caregivers, they were counted as positive errors and were considered a safety measure. However, the FN column in Table 11 indicates some cases of falling from a chair that were misclassified as sitting on a chair. After analyzing the photographs and signals of this case, as shown by the examples in Figure 13, respectively, and Figure 14, the video shots and signal patterns that represented falling and sitting looked similar and were hard to differentiate, even when judged by human eyes. This result was considered a limitation of the proposed method. Additional features, such as distance changes measured from the roof, should be considered as future work to solve this limitation. An additional limitation was that an ultrasonic signal was transmitted from a node as a triangular shape, as observed from the top view. Therefore, small areas between neighboring nodes were regarded as blind areas in which falls were impossible to detect. System designers must carefully design the number of ultrasonic nodes in the array based on the blind zone to be smaller than the minimum human width, as recommended above. This may guarantee protection against not detecting a human fall.

7. Conclusions

To prevent serious risks to elderly individuals after falling in a room, it is necessary to simultaneously monitor elderly behaviors without intruding on their privacy, detect falls, and immediately inform caregivers, when a fall occurs so they can provide urgent assistance. A design method for an ultrasonic sensor-based system is proposed in this study for elderly fall monitoring in a smart room. In this design, ultrasonic sensors are installed as a sensor array on a wall under the condition that the ultrasonic signal covers the area of the whole room with a limited blind zone. The blind zone is determined in advance to be smaller than the width of the human, and the determined blind zone and average human fall duration are used to calculate the distance between neighboring ultrasonic nodes and the total number of ultrasonic nodes on a wall. Then, activated ultrasonic nodes are transmitted in a one-by-one manner without interference in a zigzag scanning line, and the ultrasonic signals, which are time-independent, are classified as a fall or a nonfall by a time-independent-based classifier, such as LSTM. The performance of the proposed method is confirmed to be effective.

Data Availability

Data are available at https://livermutlac-my.sharepoint.com/:f:/g/personal/eleccmk_rmutl_ac_th/EsYB9mvw2ZpFiXJ9zervDcIBSBdMa3enOdK17fhf0M2-DA?e=Znch33.

Conflicts of Interest

The authors declare that they have no conflicts of interest.


The authors are thankful for PhD scholarship supported by the Rajamangala University of Technology Lanna (RMUTL), Chiangmai, Thailand (195/2557).